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MEASUREMENT AND EXPERIMENTATION IN EDUCATIONAL 
METHODS’ 
V. A. C. HENMON 
University of Wisconsin 

For three years I have been associated with the Modern Foreign Lan- 
guage Study, interested particularly in the production of prognostic 
and achievement tests as instruments of analysis and in the setting up 
of controlled experiments on objectives, curriculum-content, organization 
and method in the teaching of the modern languages. This work cover- 
ing as it did necessarily a wide range of problems has occasioned many 
hours of reflection on the possibilities and also on the difficulties of meas- 
urement and experimentation in education. The outstanding character- 
istic of our generation in education is the application of quantitative and 
experimental methods. We have taken seriously the ideal of the exact 
sciences ‘‘to measure that which is measurable and to render measurable 
that which is not.’’ The challenging axiom of one of our leaders, ‘‘ Every- 
thing that exists at all exists in some quantity and can be measured,’’ we 
have accepted, and the application of quantitative methods goes on apace. 
We teach our students to be scornful of tradition and of mere observa- 
tion and insist that all things must be subjected to the test of controlled 
experimentation. 

This is undoubtedly a healthy attitude to take if education is to be- 
come a science but the constant reader of present day educational lit- 
erature cannot in his critical moments help but be troubled by the im- 
perfections and ambiguities of our measurements and the inconclusive- 
ness of our sporadic experiments. When, for example, on such an im- 
portant problem for educational theory and practice as the effect of equal 

ictice on individual differences, whether equal practice inereases or de 
creases them, we find out of twenty-four experimental studies twelve of 
them leading at least tentatively to the conclusion that differences are 
increased and twelve to the conclusion that differences are decreased, we 
cannot help wondering about our experiments and about the conclusions 


* Address before the Eighth Annual Session of the Ohio Educational Conference 
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derived from them. Experimentalists in education perhaps need to be 
reminded of Aristotle’s observation, ‘‘Those who take in but a few con. 
siderations easily decide.’’ The temptation to play up apparently jus. 
tified conclusions, especially when they run counter to generally accepted 
notions, is too strong to be resisted and is the bane of education in 
America. Bacon’s criticism of experiment in his day applies aptly to 
much educational experiment in our day, ‘‘Even when men build any 
science and theory upon experiment, yet they almost always turn with 
premature and hasty zeal to practice, not merely on account of the ad- 
vantage and benefit to be derived from it, but in order to seize upon som 
security in a new yndertaking of their own not employing the r 
mainder of their labor unprofitably, and by making themselves conspicu 
ous, to acquire a greater name for their pursuit. Hence like Atalanta 
they leave the course to pick up the golden apple, interrupting their 
speed, and giving up the victory.’’ A more recent critic, not unsympa 
thetic with measurement and experimentation, writes, ‘‘We live nowa- 
days in American education amid a perfect orgy of statistical methodol 
ogy, much of which can make but little or no contribution to constructiv: 
pedagogy. For those who expect something Allcinseligmachend from 
these proceedings one might paraphrase from Mephistopheles: 


‘*Der Statistiker der tritt herein, 
Und beweist euch, es musst so sein: 


Das preisen die Schiiler aller Orten, 
Sind aber keine Weber geworden.’’ 

For a dozen years or more we have sharply distinguished between 
intelligence tests as measures of capacity to do the things that lie ahead 
and achievement tests as measures of what lies behind and now Kelley 
in his recent book on the Interpretation of Educational Measurements, 
has shown that this is very largely if not entirely a spurious distinction 
and that the accomplishment quotient or ratio, of which so much was 
predicted, is an unsound measure. One cannot help but think of the lines 
from Browning’s ‘‘Cleon’’ that Cattell is so fond of quoting: 


‘¢And I have written three hooks on the soul, 
Proving absurd all written hitherto, 
And putting us to ignorance again.’’ 


It may be pointed out in passing that the present almost chaotic con- 
dition in our conceptions of what psychology is about makes this quota- 
tion now peculiarly appropriate. Those who believe that psychology, if 
it is anything at all, is the science of the inner life are dubbed the medi 
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cine men of a prescientific age and beyond the pale by those to whom 
psychology is a science of behavior. Those who believe that while ‘‘psy- 
chology long ago lost its soul, is now losing its mind, but can never quite 
lose consciousness,’’ more politely dubs the extreme behaviorist as a 
biologist or as a physiologist. While the psychologist is likely to be 
scornful of education as a pseudo-science, he too has troubles of his own 
and finds his footing among biologists and physiologists pretty insecure. 

Twenty-five or more years ago, when we began to apply systematically 
the method of correlation to the study of the relationships between mental 
traits, the early results showed a surprisingly low correlation. One of 
two interpretations was possible, either our measurements of the traits 
in question were imperfect or else there is a high degree of independence 
in mental functions. The latter interpretation was accepted and has for 
a quarter of a century influenced very materially our theory of curricu- 
lum-making and methods, and our notions on transference of training. 
The reader of the accumulating literature on this important problem for 
educational and vocational guidance and for educational theory and prac- 
tice can, however, not escape the conclusion that with the improvement 
in statistical methods notably by the corrections for attenuation proposed 
by Spearman and methods of partial correlation, with the improvement 
in the reliability and validity of measurements, with the increase in the 
number of cases, control of or allowance for the variability of groups, 
and the equalization so far as possible of effects of practice, the correla- 
tions reported have steadily increased. Whatever may be the causes, 
whether due to a common central factor or to group factors, the relation- 
ships between traits are not only positive but high. Imperfections in our 
measurements not low relationship between mental traits appears from 
the accumulating evidence to have been the sounder interpretation. Our 
educational theorizing and practice have, however, been shot through 
and through and still are by notions of the .extreme specialization of 
mental functions and the non-transferability of training. In a reaction 
against discipline we have made knowledges and skills sacrosanct. Karl 
Pearson maintained years ago that ‘‘the true aim of the teacher must 
be to impart an appreciation of method and not a knowledge of facts. 
This is far more readily achieved by concentrating the student’s atten- 
tion on a small range of phenomena, than by leading him in rapid and 
superficial survey over wide fields of knowledge.’’ I believe in the sound- 
ness of both propositions but they run quite counter to the educational 
philosophy which has underlain much of our thinking on curriculum and 
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methods for two deeades. The philosophy itself grew out of imperfect 
measurements and inconclusive experiments. 

Since James made his experiments on the transfer of training, w: 
have had upwards of forty experimental studies on the different aspects 
of this central problem of education. We are about as far from a solu. 
tion as we were when we began. When the Committee on the Reorgani- 
zation of Secondary Mathematics attempted to formulate some inferences 
which might reflect the present status of disciplinary values in educatio 
in the light of experiment, there resulted seven rather vague statements 
one of which was to the effect that ‘‘the true amount of transfer from 
one field to another has not been found by experiment, on account of 
one or more of the following handicaps: 

a) The maturity or previous training of the subjects tested. 
(b) The absence from the training of the factors most favorable 


to transfer. 
ec) The inadequacy of the tests to measure the traits sought. 


We have a single important generalization out of the welter of ex 
perimentation and discussion, that there is transfer in so far as there ar 
common or identical elements, and even this Judd characterizes as vague 
ambiguous and useless. Judd has proposed a theory of generalization 
and in the literature we have two supposedly contrasting theories pre- 
sented, the theory of identical elements and the theory of generalization, 
as if generalization could mean anything else than finding identities be 
tween experiences. It is easily conceivable that of two pupils seated 
side by side under the same teacher, one because of fertility of ideas 
capacity to see similars in experiences, sagacity, mental alertness, or 
whatever we may e¢all it, will secure transfer values, the other will not 
It is also conceivable that of two teachers both good so far as any stand- 
ards of technique of instruction we now possess are concerned, one will 
make instruction widely transferable, the other will not. Transfer the 
becomes a matter of individual differences in the learner and individual 
differences in teachers and teaching. Even if we grant that there is 
transfer only in so far as there are common or identical elements, who 


knows whether the elements common to two activities or two school sub 
jects are many or few. One may claim that the common elements are 
many or important, the other that they are few or unimportant, without 
fear of successful contradition. We have not settled the quarrels about 
disciplinary values or transfer nor is there any prospect of our doing 
so. As a matter of fact Thorndike’s important study of mental disci- 
pline in high-school subjects which goes to show that while the general 
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improvement due to studies in the capacity to think is small, the differ- 
ences between studies in respect to it are likewise small, can very well 

interpreted to mean great community of elements in school subjects. 

For some months I have been coéperating with Professor Frank N. 
Freeman in a eanvass of the literature of educational psychology now 
offered to teachers in an effort to determine what facts, laws or prin- 
iples educational psychology has to offer as specific guides to practice. 
It is a rather disconcertingly difficult task. An example or two will suf- 
fice. Educational psychology might be expected to offer specific sugges- 
tions on the effect of different modes of presentation in memorizing such 
things as lists of words, whether they should be presented to the eye or 

the ear, or to both senses simultaneously, and whether the words 
should be articulated at the time of presentation. This is a very specific 
ind definite problem. Here is what a very competent text in psychology 
is to offer: 


Much experimental work has been performed to determine the relative value 


f these various modes of presentation. No consistent results have been obtained 
Some experimenters have found that articulation is an aid with the majority of 
*ts, while the opposite result has been obtained in other experiments. Some 

d that visual presentation is better than auditory for most subjects, while 
presentation is the more efficient in other experiments. Sometimes th 


al may be memorized more readily when it is presented to the eve and ear 


ltaneously, and sometimes better results are obtained when one sense organ is 

I Evidently the relative values of the several modes of presentation vary 
vith the individual, and probably with other conditions as well. 

[The individual factor is probably due in large part to differences in training 

its of study 4 person who is accustomed to articulate words while memoriz 

ng them will naturally be at a disadvantage when requested to inhibit this tendency, 

shile a person who does not articulate will be distracted by the attempt to do so 

\ l student who organizes and memorizes the materials of a lecture while 


ng to it will naturally be able to memorize better with auditory than with 
presentation, while those students who are accustomed to the textbook method 
ruction will do better with vision. It is thus obvious that any statement 
rning the relative values of any two modes of presentation is worthless unless 


fh 


subjects employed in the experiment are equally practiced in memorizing material 


these two ways.’’ 

If this is an accurate summary of the statements that can fairly be 
made, and it is, after upwards of forty experimental studies, how can 
rules of practice or any helpful suggestions for method be extracted from 
them? Similar difficulties in securing clear cut statements for the guid- 
ince of practice are encountered on such matters as the distribution and 
length of practice periods, on learning by wholes and by parts, on rate 
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of reading in learning, on retroactive inhibition, and on fatigue, to men. 
tion only some notable examples. These illustrations, and they could be 
greatly extended, may suffice to point the moral, that our measurements 
are too imperfect to base a true science of education or a methodology 
upon them, that our experiments are too inadequately controlled, and 
that educational processes are so complex that our relatively simple ex. 
periments are incapable of revealing them. When we bear in mind that 
these experiments, cited for illustrative purposes, are for the most part 
laboratory experiments and not carried out in schools and under school 
conditions, the inconclusiveness of them is the more apparent and the 
difficulties of interpretation and application become all the greater. 

Methods of instruction, we should all agree, must conform to the 
laws and principles under which the organism works most economically 
and effectively in acquiring its experiences. But it is just these facts, 
laws, and principles of human behavior that it is so difficult to formulate 
and which if they could be formulated would be the real foundations of 
method. The highly integrated forms of response even of the six-year 
old child are so complex as almost to defy analysis. Every educational 
process or product that has been studied quantitatively and experi- 
mentally turns out to be much more complex and intricate than we had 
supposed it to be. Witness the situation in the two subjects—reading 
and arithmetic—most assiduously cultivated by experimental education. 
Both bristle with unsolved problems. We are not clear as to what read- 
ing ability is nor can anyone trace out at all accurately the way in which 
our reading habits are built up. Darwin long ago observed that we do 
not know a thing until we know how it came to be and what it is. Now 
that psychology is giving so much critical study to the simplest reflexes 
and the methods of conditioning them in infants and the higher animals, 
perhaps we can hope for greater clearness as to the way in which higher 
orders of habits are integrated. 

It is easy to be critical of our achievements and to set forth our de- 
ficiencies but it is better that we who are interested in making educa- 
tion a science and founding an art upon it should be clearly conscious of 
our limitations than that others should point them out for us. The sage 
observation of Socrates that he who knows nothing knows much in know- 
ing that he knows nothing, has much of comfort in it. Courtis has recently 
rather pessimistically declared ‘‘ We have no science worthy of the name, 
and no prospect of developing one so long as we refuse to face the in- 
adequacy of our present results and the non-scientific character of our 
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many rationalizations. . . . . We have deceived ourselves into consid- 
ering our progress to be on a higher level than it really is. The chal- 
lenge of the opportunity before us is as vast as it is unperceived by 
the rank and file of those in education whose interest is to be scientific.’’ 
If this is the situation, and I think it is, what can be done about it? We 
have no lack of faith in measurement and experiment. We need much 
better measurements and more carefully controlled experiments and 
should take steps to secure them. We have forty or more experimental 
or laboratory schools and at least one experimental college but Rugg and 
Counts pointed out two years ago that in not a single one of the lab- 
oratory schools was a controlled experiment being conducted. Our so- 
called experiments are rarely experiments in the scientific sense and sel- 
dom if ever yield results that are convincing to an unprejudiced observer. 
Such an observer at once sets about picking holes in the experiment and 
has no difficulty in doing so. 

It may be urged and has been again and again since John Stuart Mill 
wrote on the logic of the moral and social sciences that experiment in 
the scientific sense is impossible in education. Thus Shorey maintains 
‘‘In the physical sciences the experimental method isolates and discovers 
the true cause by elimination. The plurality of causes and the inter- 
mixture of effects preclude this procedure in the infinitely complex social 
sciences of ethology and education.’’ To which we may reply with John 
Ray,—‘‘Much may be done though we but endeavor and nothing is in- 
superable to pains and patience.’’ The crude experimentation to which 
we now resort must, however, be replaced if we are to justify the experi- 
mental attitude and the experimental ideal in education. 

The outstanding achievement in science during the last twenty-five 
years has been the development of genetics as an exact science. East in 
his reeent remarkable book, Heredity and Human Affairs points out that, 
‘‘Geneties was born and christened because of Gregor Mendel, not be- 
cause he was such an intellectual giant that he could analyse and codify 
the complex results which had baffled his predecessors in hybridization 
work, but because he had the really brilliant idea of simplifying his ex- 
periments to the point where he was dealing with only one or two vari- 
ables at a time. When heretofore botanists had crossed plants differing 
by hundreds of characters and had been bewildered at the apparent 
chaos of their data, Mendel used varieties which differed by a single 
striking character. This lone character he followed through generation 
after generation with the carefulness of a master workman, obtaining 
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results so simple that he was able to give them their correct interpreta- 
tion.’” We need to learn this great lesson of controlling our variables 
better in educational experiment, and it cannot be too often emphasized, 
if we are to make real progress. We need to set about deliberately and 
consciously to create the means for this enterprise to control one by one 
the variables of importance. Problems of objectives to be realized and 
the conditions of their realizability, pupil capacities, curriculum content, 
methods of instruction, and standards of accomplishment are so complex 
and reveal so many uncontrollable factors or factors that can be con- 
trolled only with the greatest difficulty that only long-time, large-scale 
investigations will yield final and conclusive evidence. The difficulties 
of the case have been brought home to me very clearly in attempting 
to attack fruitfully two fundamental practical problems in the teaching 
of the modern languages—the age at which the study of a foreign lan- 
guage can most profitably be begun and how long the study must be 
continued to secure genuinely worth while results. These are problems 
on which there is such wide difference of opinion even among language 
teachers and such strong prejudices that only indubitable evidence has 
any chance at all of carrying conviction. The problems are just now 
of practical importance in connection with the developing junior high 
school and the place of the modern languages in its curriculum. The 
answer to the question of the age of beginning is so complicated by the 
purpose or aim of the student, his intellectual level, the length of time 
the study may be continued, the methods used and the social value of 
the modern languages in relation to the subjects they replace or by 
which they are replaced that convincing experiments are difficult to de- 
vise. At any rate only a carefully planned study extending over a ten- 
year period or longer could possibly tell the story for we are interested 
not only in immediate results but in the permanent outcomes after a 
lapse of time. Comparative studies of achievements of pupils beginning 
in the seventh, ninth, and eleventh grades and in the first year in col- 
lege which show generally a higher achievement in such language capaci 
ties as can be measured as we ascend the age scale is a contribution to 
the problem but not a solution. Even if a language is learned with 
greater difficulty at the earlier levels, it may be a more permanent 
and usable acquisition when so learned. 

The problem of method in the teaching of the languages has as every 
one knows been the subject of much controversy and acrimony since the 
days of Vietor. Discussion has centered about the relative merits of 
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the grammar-translation method and the direct method. To discuss 
methods on this level seems to me to lead nowhere. If a pupil expects 
to pursue the language for four years or more, one method may be the 
most economical and effective. If he studies the language for two years 
or less, as 84 percent of the pupils now do, another method may be most 
effective. The age of beginning, previous language training, objectives 
or purposes to be realized, the length of the course, and many other 
considerations determine method. Method will be among the last of the 
problems of education to be scientifically established. Until we are 
clearer about our objectives, content, and organization, we shall con- 
tinue to flit from supervised study to socialized recitations, to problem- 
projects, to contract plans, and what not. 
The only solution that I can see lies in the creation of research insti- 
tutes in each of the school subjects for the direction of experiment and 
dissemination of information in usable form. My fondest hope will 
realized if out of the Modern Foreign Language Study there may 
erow a Modern Foreign Language Research Institute to carry on the 
needed investigations on which only a beginning has been made. We 
lave reached the stage where we realize that we cannot solve our prob- 
lems by theorizing or taking thought about them. We have also reached 
the stage when we find we can’t solve anything by appointing commit- 
tees and by taking votes. We have also, I hope, reached the stage when 
we cannot rely for progress on sporadic experiments here and there by 
nterested teachers or on research by graduate students in our colleges 
nd universities. Our great endowments could expend their resources 
to better advantage in supporting such institutes than in the support 
of short time committee investigations. If each of the schools of edu- 
‘ation in which research is being carried on would concentrate its spe- 
cial efforts on some one school subject and become the center of activity 
in this field in close coéperation with the schools of its state or region, 
we might control the numerous variables and we could expect results 
which we cannot under present conditions hope to attain. As matters 
stand now the enormous increases in secondary school and college en- 
rollments have brought it to pass that those who are most capable of 
making fruitful investigations are swamped by teaching and adminis- 
tration and constructive research languishes. 
I have said little about method in teaching and should perhaps apolo- 
gize for the omission. The real foundation of method has still to be 
built. Method apart from objectives, content, and organization is an 
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abstraction without much meaning. All we can do is to dream of th 
day when principles of method that will stand the test can be formu 
lated and in the meantime live as well as we can in our dreams. We ar 
afflicted now with inadequate measurements, inconclusive experiments, 
and a realization of the complexity of our problems. We may take com 
fort that ‘‘our light afflictions, which are but for a moment, worked out 


for us a far more exceeding and eternal weight of glory.’ 
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THE RETENTION OF AMERICAN HISTORY IN THE JUNIOR 
HIGH SCHOOL! 


FOWLER D. BRooKS AND 8. JANET BASSETT 
Johns Hopkins University 

THE question of how well children retain the things they learn at 
school is of perennial interest. And, while many people believe that the 
schools are inefficient because children forget quickly so much of what 

ey learn, yet there is no valid evidence showing just how permanent 
their learning really is. The majority of opinions are based upon un- 
ritieal observation or prejudice. 

How well do pupils remember the history which they learn in junior 
high school? When do they forget most rapidly? Both of these ques- 
tions are of practical importance for teaching. If we know the amount 
which pupils forget in a given time, we have a factual basis for evaluat- 
ng the efficiency of instruction and can know if improvement is needed. 
(f we know when pupils forget most rapidly, we have a basis for deter- 
mining when reviews should be given. Current psychological opinion is 
based largely upon the classic work of Ebbinghaus on learning and re- 
learning nonsense syllables, and upon other allied investigations. A much 
etter basis is secured by ascertaining the amount of a subject known 
by pupils at a specified time and at stated intervals thereafter. Only in 

is way can adequate information be secured on the amount forgotten 
nd on the rate of forgetting. 

Two early attempts to find a solid basis of fact for this problem may 
e cited. Along in the spring of 1900 a committee was appointed to 
nvestigate (survey!) the publie schools of the District of Columbia. 
[he chief examiner of the Civil Service Commission prepared questions 
m the arithmetic, history, ete. taught in the grammar grades. These 
questions were given to 1188 first-year high-school pupils nine months 
fter they had finished the grammar school. The arithmetic scores aver- 
ged 58.8 percent and the history scores 53.1 percent. The investigation 
committee concluded that nine months after leaving the grades more than 
{0 percent of the grammar school content could not be recalled,? although 


* The material of this report is part of the Ph. D. dissertation of S. Janet Bassett 
at Johns Hopkins University, 1928. Other parts relate to sex differences in retention, 
factors influencing retention, and the kinds of history knowledge most readily for- 
gotten and the kinds retained best. 

* Supplemental Report, School Examinations, Apr. 14, 1900, Senate Report 711 
part 2, 56th. Congress, First Session, 15 pp. 
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there was no evidence to warrant the assumption that the pupils ever 
did know all the things called for by the questions or ever could have 
made perfect scores on them; in fact, we are certain that these pupils 
never were able at any time during their grammar-school days to make 
average scores of 100 percent on the tests given. 

A little later another investigation® of the amount of subject matter 
recollected by 1362 high-school students from one to five years after it 
had been studied shows that only 5 of 33 questions on history and 
geography were correctly answered by as many as 75 percent of the 
pupils. Here again, no information was available to show the amount 
which the pupils did know at any specified time previous to giving the 
tests to determine how much was remembered. 

Recent investigations* of the permanence of school learning (exclud 
ing, of course, those to ascertain the effect of the summer vacation) do 
not have data showing the amount which students actually did know 
when they quit studying a subject at school, but infer the amount from 
other sorts of information, such as the scores made by other similar 
groups of pupils who have just finished the subject. 

The present report shows the amount of American History actually 
retained by 495 students in Grades VII and VIII of several publie schools 
in Baltimore from four to sixteen months after the semester in which it 
was studied.® The results indicate that pupils forget 23 percent of first 
or second-year junior high-school American History in a year’s time. 

The Baltimore city course of study in American History in each half 
vear of Grades VII and VIII is a distinct unit with no overlapping 
Standardized history tests could not be used to measure retention, be 
eause they do not eover fully the subject matter actually taught in each 
half year. Accordingly, four comprehensive new-type examinations were 
prepared to cover the half-years, VIIB, VILA, VIIIB, and VIIIA, respec 


tively. Preliminary try-outs were made upon 187 pupils not otherwise 


*Cober, E. W. ‘‘A Study of High-School Pupils with a View of Determining 


the Extent of Recollection of Once Familiar Facts.’’ Thesis, University of Pennsy! 
vania, 1912. 46 pp. 

‘For example, Eikenberry, D. H. ‘‘ Permanence of High School Learning.’ 
Journal of Educational Psychology, Vol. XIV (1923), pp. 463-481; Powers, S. R 
A Diagnostic Study of the Subject Matter of High School Chemistry. (New York 
Teachers College, Columbia University, 1924), pp. 49-54; Thorndike, E. L. ‘‘ The 


Permanence of School Learning.’’ School and Society, Vol. XV (1922), pp. 625-627. 


* Investigations on the retention of other subjects have been under way for tw 
or three years by graduate students at Johns Hopkins University. 
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ised in the investigation. The tests were revised in the light of the results 
from these try-outs and upon the combined judgments of history teachers 
nd supervisors and several competent students of experimental educa- 
m. In their final form, each of the tests consists of four parts,—40 true- 
false statements, 10 completion items, 20 multiple-choice elements, and 
+ modified essay questions. On the latter, the student selected any three 
the four questions and gave four statements to answer each of the 
ree selected. A perfect score on each test is 82. 
A part of the test used in Grade VIIA is given to indicate the nature 
the tests. 
AMERICAN HISTORY TEST, GRADE VIIA. 
Copyright, 1928 by 8. JANET BASSETT 
Part I 
Directions: Some of-the statements below are true and some of them are false. 
on the line at the left of a statement if the statement is true. Put on the 
at the left of a statement if it is not true. Go through the list of statements 
‘kly, answering those of which you are sure, then go back and study the more 
ficult statements. Answer these if you think you know them. 
l. Sample. 
3. Hamilton strongly objected to Jay’s Treaty. 
4. For a long time the Barbary pirates in northern Africa preyed on the 
commerce of Christian nations. 
The Napoleoni Wars stopped all American commerce. 
One of the causes of the War of 1812 was that the British searched 
American vessels. 
Spain sold us Louisiana. 
We wanted Louisiana in order to secure an outlet to the Pacific Ocean. 
Florida was explored and settled by Spain. 
Florida was used as a naval base by England in 1812. 
Mexico granted large tracts of land in Texas to settlers from the United 
States. 
The West was rapidly settled after the discovery of gold in California. 
Few immigrants have gone beyond the Appalachians. 
The Indians of the Oregon region welcomed the White Man. 
The acquisition of so much land in the West helped to do away with the 
slavery dispute. 
Before the Civil War there was a lack of public schools in the Southern 
States. 
41. The Erie Canal stopped the trade between the West and the South. 


Part II 

Directions: Some important words have been omitted from the following state 
nts. Write in the blank the word that best completes the meaning of the sentence 
h blank represents one word. 

Sample. 

The hero of New Orleans was 

The two kinds of tariff are and 

The ‘‘ Pathfinder’’ was ? 

Mexico claimed that the boundary line between Mexico and Texas was the 


The industrial growth of the was largely dependent upon 
**King Cotton.’’ 
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Part III. 


Directions: In each sentence select the one word or phrase which you think makes 

the statement true. Underline this answer. 

1. Sample. 

2. England seized our sailors for her ships on the ground that they were (a 
deserters from the English army; (b) good fighters; (c) subjects of England 
(d) enemies of England. 

4. ‘*Don’t give up the ship,’’ was said by (a) Hull; (b) Jones; (c) Decatur; 

(d) Lawrence. 

Tariff is (a) a veto; (b) a debate; (c) an amendment; (d) money paid on 

imported goods. 

10. An important effect of the Louisiana Purchase was that it (a) helped Jeffer 
son to continue his doctrine of ‘‘strict’’ construction; (b) made Jefferson 
very unpopular; (¢) gained the friendship of Spain; (d) gave the United 
States control of the Mississippi. 

16. One of the great hindrances to western development was the (a) inventioz 
of the steamboat; (b) danger from the Indians; (c) increase in immigra 
tion; (d) discovery of gold. 

17. The last frontier was (a) along the Pacific coast; (b) along the Gulf of 
Mexico; (c) on the banks of the Mississippi; (d) at the eastern base of 
the Rockies. 

21. A great American writer was (a) Patrick Henry; (b) Henry Hudson; (c 
Dred Scott; (d) Henry Wadsworth Longfellow. 


Part IV. 


Directions: Select any three of the following problems about which you know 
the most important facts. Write four statements on each subject in the space which 
is left beneath the topic. Think before you write. 

1. How did we gain our freedom on the sea? 

2. What was the tariff policy of the United States in 1860? 

3. Why was the Louisiana Purchase the greatest event in Jefferson’s admit 

istration? 

4. How has the Industrial Revolution improved American life? 


By repeating the tests with 72 pupils, not otherwise used in the in 
vestigation, the practice effect was found to be small, and the reliabili 
ties were found to range from .64 to .90 when the tests were repeated 
at intervals of one week to eight weeks. The pupils had taken a great 
many standardized tests and new-type examinations, so that we would 
expect little practice effect from repeating these tests, especially at inter 
vals of four or more months. 


The tests, in final form were given according to the following sched- 
ule. The VIIB test to the VIIB grades at the close of the semester and 
thereafter at intervals of four, eight, twelve, and sixteen months. The 
VIIA test to the VIIA grades at the close of the semester (and thereafter 
at intervals of four, eight, twelve, and sixteen months.) And similar) 
in Grades VIIIB and VIIIA. Thus we have data showing how much of 
the history taught during a semester was known at its close, and how 
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much was known at intervals of four, eight, twelve, and sixteen months 
ikes thereafter. The results are shown in Tables I and II. 


TABLE I 








{a 
nd AMERICAN History Scores In GRADES VIIB, VIIA, VIIIB, anp VIIIA AT THE 
CLOSE OF THE TERM IN WHICH IT WAS STUDIED AND AT INTERVALS OF 
ur Four, E1egut, TWELVE, AND SLXTEEN MONTHS THEREAFTER 
ar American History Scores 
NUMBER At close After intervals of 
(;RADE or Oo A —_——_ 
Ter Pups term 4 mo 8 mo 12 mo 16 mo 
801 VIIB 167 Mean 50.2 43.7 
‘ 8. D 13.2 11.5 
- 107* Mean 55.1 47.3 43.8 
8. D 12.5 10.9 10.6 
™ 64" Mean 56.6 47.6 44.7 40.7 39.0 
:, 8. D 13.9 11.8 11.5 11.3 11.2 
‘Ta 
VIIA 142 Mean 59.9 53.8 50.2 
Ss. D 9.6 10.2 9.7 
of 120® Mean 61.1 54.4 50.6 48.6 
7 8. D 9.8 10.7 10.2 10.6 
- 86 Mean 60.1 51.5 
8.D 9.5 8.9 
¢ VIIIB 79 Mean 61.9 50.5 
8. D 10.4 11.2 
44° Mean 58.4 47.3 44.1 
8. D. 11.2 12.2 12.6 
VIIIA 21 Mean 54.7 49.5 46.3 
8. D 6.3 6.7 6.3 
7 
} *Included in the group of 167 pupils of Grade VIIB who were retested only once; that is, after an 
nterval of 4 months 
bIncluded in group of 142 pupils of Grade VIIA who were retested after intervals of 4 and 8 months 
*Included in group of 79 V IB pupils who were retested after an interval of 4 months 
i TABLE II 
PERCENTAGE OF AMERICAN HISTORY KNOWN AT THE END OF THE SEMESTER (IN WHICH 
IT WAS STUDIED) WHICH IS FORGOTTEN Four, E1Gut, TWELVE, AND SIXTEEN 
MONTHS THEREAFTER (GRADES VIIB To VIIIA) 
PERCENTAGE oF History ForGorrTeN IN 
GRADE NUMBER OF —- —--— —-— —-- —-— -— -- 
PuPILs 4 mo 8 mo 12 mo 16 mo 
VIIB 167 12.9 
VIIB 107" 14.3 20.5 
VIIB A4e 15.9 20.9 28.0 31.0 
VIIA 142 10.2 16.3 
VIIA 120% 11.1 17.2 20.5 
VIIA 86 14.3 
VIIIB 79 18.4 
VIIIB 44¢ 19.0 24.5 
VIIIA 21 9.6 15.4 
d » b- © See corresponding footnotes to Table I 
An examination of Tables I to III shows that forgetting is most rapid 
ne during the first four months after the close of the term in which the 
er history is learned, and that thereafter it is much slower. Data on other 
i individuals indicate that pupils at the beginning of a semester knew very 
0] ittle of the history which was taught during that term, their scores 
w being, on the average, approximately 10 points on the appropriate tests 
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TABLE III 
PERCENTAGE OF AMERICAN HISTORY KNOWN AT THE END OF THE SEMESTER (IN WHICH 
IT WAS STUDIED) WHICH IS ForRGOTTEN Four, EIGHT, TWELVE, AND SIXTEEN 
MONTHS THEREAFTER (DATA COMBINED FROM GRADES VIIB, 
VIIA, VIITB, anv VIITA) 


PERCENTAGE ForGOTTEN AFTER 


NUMBER oF —_—__— $$ —___ 
Pupts 4 mo 8 mo 12 mo 16 mo 
409 12.9 
314 12.8 18.8 
1S4 12.8 18.5 23.1 
64 15.9 20.9 28.0 31.0 


Table III is to be read as follows: 409 pupils in Grades VII and VIII forget 12.9 percent of the histor 
which they knew at the close of the term in which they had studied it; 314 pupils forget 12.8 percent in 4 
months and 18.8 percent in 8 months; ete. 


devised and used in this investigation. The results are approximately 
us follows: one-eighth is forgotten in four months, one-sixth in eight 
months, one-fourth in twelve months, and one-third in sixteen months 

The pupils who made high seores at the end of the term usually mad 
high seores four, eight, twelve, or sixteen months later, and those wh 
made the lower scores the first time they were tested usually made the 
lower scores at the subsequent retestings, as indicated by the correlations 
between scores at the end of the semester and those at succeeding inter- 
vals (Table IV), the coefficients ranging from .74 to .89 and averaging 
more than .80. Furthermore, those who rank high at the end of four 
months are likely to rank high at the end of eight months, the correla 
tions ranging from .90 to .97 and averaging well over .90. This is not 


TABLE IV 
CORRELATIONS (PEARSON) BETWEEN AMOUNTS OF AMERICAN HISTORY KNOWN 
THE END OF A TERM AND THE AMOUNT KNOWN Four, EIGHT, TWELVE, AND 
SIXTEEN MONTHS THEREAFTER (GRADES VII AND VIIT) 


INTERVAL 

GRADE NUMBER OF - icmemnaet - 

PupiIts o-4* 0-8 0-12 0-16 4-8 
VIIB 167 . 887 
VIIB 107* 875 843 914 
VIIB 646 891 869 811 807 953 
VIIA 142 . 859 787 909 
VIIA 120» 878 826 $12 934 
VIIA 86 834 
VIUIB 79 , 
VIIIB 44° 77! 768 .974 
VIIIA 21 .775 .740 899 


*O means at close of term in which the history tested was studied 

sb. ¢ See corresponding footnotes to Table 

Table IV is to be read as follows: The correlations between the history scores of 167 pupils in Grade 
VIIB at the close of the term and their scores 4 months later is .887, etc 


surprising; in fact, it is exactly what we would expect, since the most 
rapid forgetting takes place during the first four months after ceasing 


the study of any semester’s work in the subject. 
While the results of this investigation are in agreement with the 
widely accepted view that those who know the most also retain the most 
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true,—that is, that those who know the least forget the most. The first 
statement does not necessarily imply the second. As a matter of fact, 
» the data of this study indicate that the pupils who know most at the 
end of the term tend to forget a little more during the ensuing months 
than those who make the lower scores at the first testing. The correla- 
tions between the scores at the first testing and the amounts actually for- 
gotten during the succeeding months range from .022 (for 120 pupils 
in Grade VIIA retested twelve months after the initial testing) to .604 
for 64 VIIB pupils retested after sixteen months), they average .36 
lable V). There is no contradiction here. The pupils who know most 
it the end of the semester tend to forget a little more than those who 
know less, but they still know the most, preserving very much the same 
relative standings as on the initial test. 
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TABLE V 


CORRELATION BETWEEN History SCORES AT THE CLOSE OF THE TERM AND THE 
AMOUNTS OF History Forcotrren DvurInGe THE ENsuine Four, EImGnt, 
TWELVE, AND SIXTEEN Montus (Grapes VII anv VIII) 


Pearson Corre.ations Between Scores at Crosz 
or Term anpd Amounts Forcorren 
Dcrinc INTERVALS OF 





Grape NUMBER OF — 
Pourits 4 mo 8 mo. 12 mo 16 mo 
VIIB 167 . 520 
VIIB 107" .523 . 526 
VIIB fi4e . 529 .486 . 589 . 604 
VIIA 142 . 156 . 291 
VIIA 120% .086 . 241 022 
VIIA 86 389 
VIIIB 79 . 200 
VIIIB 44° . 223 . 185 
VIIIA 21 . 249 . 352 


* & © See corresponding footnotes to Table I. 


CONCLUSIONS 

From the data of this investigation the following conclusions may 
be drawn: 

1. In a year’s time junior high-school pupils forget 23 percent of 
the American History which they knew at the close of the semester dur- 
ing which it was studied. 

2. Forgetting is twice as rapid during the first four months after 
the semester in which the history material was presented as it is during 
the next four months; and it is slower thereafter. 

3. Accordingly, reviews should be continued at least during the ear); 
part of the semester following the one in which the material is presented. 

4. Pupils who know most tend also to retain most at later intervals, 
individual ranks being quite similar at the initial testing and the later 
retestings. 

5. Pupils who know most tend also to forget a little more than those 
who know less, but they still maintain relative standings at later retest- 
ings quite similar to those on the initial ones, the average correlations 
being more than .80. 

6. Accordingly, the best students need reviews as much as do the 
average or poorer students. 

7. After the fourth month of forgetting pupils make little change in 
relative standings, all the correlations being .90 or more. 

8. Whenever courses of study in the junior high school provide sepa- 
rate and distinct units of work for each half year, definite provision is 
needed for reviewing each term’s work during the following term. In 
this way permanence of learning can be more effectively secured. 
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THE VOCABULARY LOAD OF CERTAIN SECONDARY SCHOOL 
MATHEMATICS TEXTBOOKS 


H. H. REMMERS and A. GRANT 
Purdue University 


PURPOSE 
THE purpose of this study was twofold: (1) to measure the vocabu- 
lary burden of secondary mathematics textbooks, and (2) to determine 
the reliability of measurement in the methods employed. It is assumed 
that the ease with which the pupil acquires the concepts of a subject is 
a function of the textbook’s vocabulary load, and that therefore a meas- 
urement of this load would constitute an expository device of value in 
selecting a textbook. 
METHOD 
The method used was for the most part identical with that used by 
Bertha A. Lively and S. L. Pressey in their work on ‘‘A Method for 
Measuring the Vocabulary Burden of Textbooks.’ Such variations as 
ave been introduced will be noted and the reason for their use explained. 
The first problem was to secure such sampling of words as would war- 
rant statistical inference relating to the entire book. One thousand words 
were taken as probably adequate to constitute a sampling, and also as 
convenient to work with. To obtain such a sampling, the number of 
pages in the book was first noted, with the approximate number of words 
per line; the number of pages which should be sampled, taking one line 
on each page, was then found. The pages were so chosen as to insur 
even distribution throughout the book. All the words in these lines 
were then gone over and the different words listed alphabetically 
The thousand words having thus been noted, the total number of 
different words per thousand was then found. This gave the vocabulary 
range. These words were then looked up in the Thorndike Word Book, 


* Educational Administration and Supervision. Volume IX, October, 1923. 

* Thorndike, E. L. The Teacher’s Word Book. New York City: Teachers’ Col 
lege, Bureau of Publications, Columbia University. This book lists the ten thousand 
most common words of the English language, as determined on the basis of an elab 
orate investigation by Professor Thorndike. In the Word Book each word is fol 
lowed by an index number indicative of its commonness. Thus such a common word 
as and has an index number of 210; a relatively uncommon word like atom has an 
index number of 4; still more rare words such as neolithic do not appear in the word 
book at all—those are listed in this study as zero value words. Words with credit 
numbers of 49 or over occur in the first thousand words, in frequency. Words with 
index numbers of 10 or over oceur in the first five thousand words—and so on; for 
a more detailed statement, the reader is referred to the Word Book itself. 
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and the index number of each found. This number indicates the com- 
monness of the word as found by Thorndike. The number of words not 
appearing in Thorndike’s Word Book was then listed. Since these do 
not appear, they are not among the ten thousand most common words, 
and ean be regarded as having zero for their index value, and as consti- 
tuting the highly technical part of the vocabulary. The median index 
number was then computed. This is merely a numerical index of the 
book’s vocabulary difficulty. Evidently, the lower the index number, 
the more difficult the book. (See Fig. 3). Here it should be noted that 
in computing the index number, Lively and Pressey counted the zero va! 
ued words twice. However, since this does not change the relation be. 
tween index numbers, we considered this unnecessary. 

Twelve different books were thus studied, as follows: Avery, Royal A 
Plane Geometry. (New York: Allyn and Bacon, 1925); Hart, Clara A., 
and Feldman, Daniel D. Plane Geometry. (New York: American Book 
Co., 1911); Smith, D. E. Essentials of Plane Geometry. (Boston and 
Chicago: Ginn and Co., 1923); Sehultze, Arthur and Sevenoak, F. L 
Plane and Solid Geometry. (New York: Macmillan Co., 1907); New- 
ell, M. J., and Harper, G. A. Plane and Solid Geometry. (Row, Peter- 
son & Co., 1915); Wentworth, George and Smith D. E. Plane and Solid 
Geometry. (Boston and Chieago: Ginn and Co., 1913); Ford, W. B 
and Ammerman, Charles. First Course in Algebra. (New York: Mac- 
millan Co., 1921); Ford, W. B., and Ammerman, Charles. Second 
Course in Algebra. (New York: Maemillan Co., 1920); Wells, Webster 
Algebra for Secondary Schools. (Boston and Chicago: Heath & Co., 
1908); Young, J. W. B., and Jackson, L. L. Elementary Algebra First 
and Second Course. (New York: Appleton & Co., 1910); Wells, 
Webster and Hart, Walter W. New High School Algebra. (Boston and 
Chieago: Heath & Co., 1912): and Smith, D. E., and Reeve, W. D 
Essentials of Algebra. (Boston and Chicago: Ginn and Co., 1924). 

Tables I, If and III summarize the findings. For each book two 
counts were made to compare the observed with the theoretical variabil- 
ity. The method used for the second count was identical with that for 
the first count, except that a different page was used. The probable 
errors were found for the vocabulary range and for the zero words. In 


both eases this is done by use of the formula’ P. E. 6745 4| PQ where 


P y 


‘Kelley, T. L. Statistical Method. Pages 89-90, on the standard deviation « 
a proportion. 
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is the number of words (1,000), P is the number of successes (i. e., in 
this ease different words), and Q the number of failures (i. e., N minus 
P). The application of the formula may be illustrated with the first 
eount from Smith (Table I): P probability of suecess (220 


Q = probability of failures (1,000 minus 220); N= 1,000; 


220 XK 780 
= 1 


8.82 


TABLE I 
NUMBER OF DIFFERENT WoRDS FOUND ON Two COUNTS IN CERTAIN SECONDARY 
ScHooL TEXTBOOKS IN MATHEMATICS 


First P.E SECOND P.E Dirr P_E Dirr. 

Count Count diff P.E. aut 
Smith 220 8.82 225 8.88 5 12.48 0.40 
Avery 234 9.02 243 9.14 9 12.83 0.72 
Hart & Feldman 248 9.21 236 9.02 12 12.88 0.93 
Newell & Harper 260 9.34 272 9.51 12 13.30 0.90 
Wentworth & Smith 274 9.52 283 9.55 9 13.46 0.67 
Schultze & Sevenoak 301 9.76 312 10.02 11 13.95 0.79 
Smith & Reeve 251 9.00 270 9.49 19 13.07 1.45 
Ford & Ammerman I 277 9.52 290 9.66 13 13.54 0.96 
Ford & Ammerman II 320 9.96 332 10.06 12 14.11 0.85 
Wells & Hart 300 9.76 325 10.00 25 13.95 1.79 
Wells 342 10.10 354 10.18 12 14.33 0.80 
Young & Jackson 380 10.38 362 10.24 18 14.58 1.23 


This table is to be read as follows: The first 1,000 word count for Smith yielded 220 + 8.82 different 
words; the second count, 225 + 8.88 different words. The difference of the countsis5 + 12.48, which 
s statistically not significant 


TABLE II 
ZERO WORDS 


Finst 8.D SEconpD $.D Dirr P.I Dirt 

Count Count a PLE. dir. 
Smuitt 23 4.54 25 4.70 2 4.40 0.45 
Avery 33 5.32 34 5.41 1 5.12 0.20 
Hart & Feldman 34 5.41 27 4.90 7 4.92 1.42 
Newell & Harper 56 6.62 45 6.13 11 6.08 1.62 
Wentworth & Smith Od 6.57 57 6.72 3 6.33 0.47 
Schultze & Sevenoak 51 6.50 62 7.05 11 6.45 1.71 
Smith & Reeve 40 5.79 48 6.27 8 5.75 1.39 
rd & Ammerman I 36 5.58 36 5.61 0 5.33 0.00 
i & Ammerman II 31 5.21 45 7.25 14 6.01 2.34 
Wells & Hart 40 5.86 48 6.39 s 5.85 1.37 
Wells 49 6.59 54 6.76 5 6.36 0.79 
Young & Jackson 46 5.31 49 6.50 3 5. 65 0.53 


This table is read in the same way as Table I. 


This result is to be interpreted to mean that with further samplings 
from this’ book it is an even wager that on the average the values for 
the category of ‘‘different words” will go as high as 228.82 and as low 
as 211.18. This application of the theory of sampling constitutes an 
important check on the reliability of the counts, and hence on the 
method used. 
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RESULTS 

Range of Vocabulary.—The results are best shown in graphic forn] 
Figure 1 shows the difference in range. It is evident that books van P 
widely, 220 to 380 different words per thousand, with respect to ead 
other. The range within books is, however, small when compared wit! 
ranges found in other familiar books, by Lively and Pressey. For ej 
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ample, Muzzy’s American History has a range of 533; Thackeray’s Va 
ity Fair, 490; and Horace Mann’s Second Reader, 412. It seems thé 
the most difficult vocabulary burden in the books investigated with r§ 
spect to range does not equal that of Mann’s Second Reader. Evident! 
the same words are used over and over again, and the vocabulary use] 
is not large. 

Number of Zero Words.—Here again graphical representation is usé 
The range in this ease is also wide, being 23-62. Comparison with tl 
results found by Lively and Pressey are again interesting. Muzzy! 
American History has 27 zero words, and McPherson and Henderson § 
General Chemistry has 68. This latter number is higher than the num 
ber of zero words in the most difficult of the mathematics books invest! 
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Fig. @ The number of zero words for hve samplings of woo words 
foraach book 


gated. It is evident that the number of technical words in some other 
high-school books is higher than that found in the mathematics books. 
Evidently the same words are used often. 

Since the number of zero words represents the size of the technical 
vocabulary, these numbers may be used as a basis of comparison. Before 
this is done, however, it is necessary to take into account the errors which 
each involves. Since these measures are unrelated, the probable error 
of the difference of two averages, due to variable errors of measurement 
and sampling, is given by the formula :* 


*Kelley, T. L. Statistical Method. Page 182 on the standard deviation of a 
difference. 
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P.E. pitrerence = A PE tevorace + P.E.* erage . 
I u 


With these computed it is possible to compare any two books. Compari- 
sons were made among different geometry books, different algebra books, 
and between geometry and algebra books in general. These comparisons 
are listed in Tables II and III. It will be noted that since the number 
of zero words is small, the P. E. is correspondingly large. 


TABLE III 
INTER-COMPARISONS OF VARIOUS TEXTBOOKS IN GEOMETRY 





T'exTsoox Zero Worps 8.D Dirr P.E.ag _ Dirr 
P.E. aig 
Smith 23 4.54 10 4.71 2.12 
Avery 33 5.32 
Smith 23 4.54 ll 4.76 2.31 
Hart & Feldman 34 5.41 
Newell & Harper 56 6.62 a3 5.31 6.22 
Smith 23 4.54 
Smith 23 4.54 28 5.34 5.25 
Schultze & Sevenoak 51 6.50 
Smith 23 4.54 31 5.38 5.76 
Wentworth & Smith 54 6.57 
Avery 33 5.32 21 5.70 3.68 
Wentworth & Smith 54 6.57 
Avery 33 5.32 23 5.72 4.03 
Newell & Harper 56 6.62 
Avery 33 5.32 18 5. 66 3.18 
Schultze & Sevenoak 51 6.50 
Hart & Feldman 34 5.41 22 5.76 3.82 
Newell & Harper 56 6.62 
Hart & Feldman 34 5.41 20 5.74 3.48 
Wentworth & Smith 54 6.57 
Hart & Feldman 34 5.41 17 5.70 2.98 
Schultse & Sevenoak 51 6.50 
This table is to be read: The first 1,000 word count of Smith yielded 23 zero words with a 8.D 

4.54 Avery yielded 33 with a S8.D. of 5.32. The difference is 10 + 4.71. This indicates no signifi 


cant difference in the difficulty of the two books 


Assuming that three probable errors constitute a significant differ- 

ence, the following conclusions may be drawn: 

AmonG GEOMETRY BOOKS. 

1. Smith, Avery, and Hart and Feldman all have a significantly smaller technical 
vocabulary than Schultze and Sevenoak, Wentworth and Smith, and Newell 
and Harper. 

2. The largest difference exists between Smith and Wentworth and Smith, and 
between Smith and Newell and Harper. 


AMONG ALGEBRA BOOKS. 
1. There is no real, significant difference between any of the algebra books. 
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AmONG GEOMETRY AND ALGEBRA BOOKS. 

1. All geometry books investigated have a larger technical vocabulary load 
than Smith’s Algebra. 

2. Avery has a smaller load than Young and Jackson, or Wells. 


Median Index Number.—These numbers are shown in Figure 3. 
Schultze and Sevenoak’s Geometry again shows up as the most difficult, 
aving an index number of 24. Avery’s Geometry has the lightest load, 
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aving an index number of about 60. It will be noted again that the 
ower the index number, the more difficult the book’s vocabulary load. 

Reliability of the Method Used.—The P. E.’s of the various categories 
studied. the number of different words, the zero word differences among 
different books and differences between two consecutive counts indicate 
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the variability expected on the basis of the theory of sampling. A com- 

parison of these values with the observed values in terms of the ratio 
Diff . . . *-* 

P.E. cw indicates that the theoretical and empirical data agree. It will 

be noted from Table IV that the rank-order correlation between media: 

index numbers and number of zero words (both averaged for the two 

counts) is .81. 


TABLE IV 
RANK-ORDER CORRELATIONS FOR THE CATEGORIES INDICATED 
Zero Worps Mepian Dirricv.trr 
Zero word 
Median difficulty -81 
Different words 4 60 


This warrants the statement that either criterion may be used with 
out danger of serious error. None of the shifts in rank are particularly) 
significant as may be seen from an inspectional comparison of Figures 2 
and 3. The correlations for the number of different words with the 
number of zero words and the median difficulty, while both positive and 
significant, are not high enough to warrant the conclusion that they maj 
be used interchangeably as measures of vocabulary load. 
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A STUDY OF A MODIFIED FORM OF THE MULTIPLE- 
RESPONSE TEST 
Francis D. Curtis 
Unwersity of Michigan 
and 


GERALD G. Woops 
University of Michigan High School 


THE purpose of this investigation was to determine some of the rela- 
tive merits of the conventional multiple-response test having five choices, 
as compared with a modified form of this test. This modified form con- 
sists of the conventional form with a blank added after each exercise. 
lhe pupils are instructed to underline the correct response if it is pres- 
ent among those given, or, if the correct response is not given, to write 
it in the blank provided. 


METHOD 

Two mimeographed sheets of test items were prepared for each class. 
Sheet I contained seventy-five items of the conventional multiple- 
response type having five choices. Sheet II contained sixty of the same 
seventy-five items which appeared on Sheet I, and fifteen additional 
items. The sixty were unchanged in wording, but to make them seem 
different from the same items on Sheet I, their order and that of the 
five responses in each item was changed; each, moreover, was made into 
the modified form by providing a blank at the end. The fifteen other 
test items on this sheet were of the modified form, but none contained 
the correct response among those given, a fact which was intended to 
be obvious to the pupil. These fifteen items were distributed at random 
imong the other sixty in order to furnish a proper mind-set, since the 
pupil saw clearly that he must write the proper response in the blank 
at the end of some of the items on this sheet. 

The subjects were 206 pupils in the University of Michigan High 
School, grouped in six classes: one seventh- and one eighth-grade class 
in general science, one ninth-grade class in general biology, and one 
tenth-grade elass in advanced biology’ all enrolled in the regular winter 
session of 1926-27: and two eighth-grade classes in general science en- 
rolled in the summer session of 1927. 

*The authors wish to make grateful acknowledgment of the help of Mrs. Grace 


K. Gelder, of the University High School Department of Biology, in preparing, ad- 
ministering, and scoring the tests in biology. 
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Both Sheets I and II were distributed to the members of each class 
during the same examination period, one sheet being given to the pupil 
after he had completed and handed in the other. In order to equalize 
practice effects, Sheet I was given first to three classes, and Sheet II first 
to the other three. 

The directions upon Sheet I were as follows: 

In the following statements, underline the one of the five words or phrases 
which alone will make the statement true. Thus, if the statement is this, ‘‘ This 
month is February, March, April, May, June,’’ you would underline June, which 
is the only one of the five words which will make the statement true. After you had 
marked it the statement would look like this, 

‘*THIS MONTH IS FEBRUARY, MARCH, APRIL, MAY, JUNE.’” 

UNDERLINE ONE WORD OR PHRASE FOR EVERY STATEMENT, BUT 
DO NOT UNDERLINE MORE THAN ONE WORD OR PHRASE FOR ANY 
STATEMENT. Be sure to mark every statement. If you are in doubt regarding 
which is the correct word or phrase to underline, guess. 


The directions upon Sheet II were: 

The following statements are of two types. Each contains five words or phrases 
followed by a blank space. In some of these statements one of the five words or 
phrases makes the statement correct. Thus: 

‘*THIS MONTH IS FEBRUARY, MARCH, APRIL, MAY, JUNE, ——— 


With statements of this type, you would underline the word which would make 
the statement correct, and would write nothing in the blank space. When you had 
finished completing the statement it would look like this: 

‘*‘THIS MONTH IS FEBRUARY, MARCH, APRIL, MAY, JUNE, ————,,”’ 
because you would have underlined June,’ since June is the only one of the five 
words which makes the statement true. 

In some of the other statements none of the given words or phrases makes the 
statement true. Thus: 

‘*‘THIS MONTH IS FEBRUARY, MARCH, APRIL, MAY, JULY, ————.”’ 

With statements of this type you would write in the blank space the necessary 


word, or as many words as are necessary to make the statement correct. In the | 


statement given here you would write June‘ in the blank, thus: 


‘THIS MONTH IS FEBRUARY, MARCH, APRIL, MAY, JULY, JUNE.”’ 

IN EACH STATEMENT THAT FOLLOWS DO NOT UNDERLINE ANY 
WORD OR PHRASE IF YOU FILL THE BLANK, BUT BE SURE EITHER TO 
UNDERLINE A WORD OR PHRASE OR FILL THE BLANK FOR EACH STATE 
MENT. DO NOT FILL THE BLANK UNLESS NONE OF THE FIVE WORDS 
OR PHRASES GIVEN MAKES THE STATEMENT CORRECT. 





* June was changed to August for the summer-school classes. 
* Ibid. 
* Ibid. 
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MODIFIED FORM OF MULTIPLE-RESPONSE TEST 


It will be noted that the sole difference in the directions for the two 
test sheets is that Sheet I requires the pupil only to decide which of the 
given words or phrases makes the statement correct, and then to under- 
line this word or phrase; whereas, with Sheet II, he must not only de- 

de whether the correct response appears in the statement, but must 
inderline it if it is present, and if not, must write it in the blank. 

The findings from the investigation were based upon the pupils’ re- 
sponses to the sixty identical items common to their respective Sheets I 

nd II; their responses to the remaining fifteen items on each sheet 
were disregarded entirely in so far as the investigation was concerned. 
FINDINGS 

lable I seems to indicate that the modified form requires only slightly, 

iny, longer to administer than the multiple-response. It will be noted 
that each elass required more time in completing whichever form was 
dministered to it first. Thus, with the three classes which were given 

e multiple-response first, the 1214 percentile pupil of the seventh-grade 
TABLE I 


ME IN MINUTES AND SECONDS REQUIRED BY THE 1214 PERCENTILE PUPIL, THE 
MEDIAN PUPIL, AND 87% PERCENTILE PUPIL TO FINISH EACH FORM OF THE TES1 


Mu ttTrPLe-REsPoNnse Mopiriep Form 

12% Per- Median 87% Per- 12% Per- Median 87% Per- 
CLASSES centile Pupil centile centile Pupil centile 
Pupil Pupil Pupil Pupil 

(Min.-Sec.) (Min.-Sec Min -Sec (Min.-Sec.) Min.-Sec.) (Min.-Sec.) 
Vil 18—20* 21-50 30-00 15-35 18-40 24-20 
Vill 17-00 24-30 32—00 15-15 19-50 25-20 
18-20 22-50 30-00 16-45 21-15 28-30 
X 17-20 19-45 25-20 24-30 32-10 37~30 
Villani 11-30 15-50 18-30 19-46 27-30 36-01 
Villese 11-00 13-45 20-30 22-00 28-30 43-00 
All classes mbined 15-40 21-30 29-25 16-10 21-05 31-45 
Weighted averages 16-47 21-24 28-20 17-38 22-35 29-25 


*The italics indicate the test-form which was administrated first to each respective class. 


shed the multiple-response and the modified form in respectively 
IS minutes 20 seconds, and 15 minutes 35 seconds; the median pupil in 
he eighth-grade finished the multiple-response and the modified form 


respectively 24 minutes 30 seconds, and 19 minutes 50 seconds; the 


ep. MoH, , ‘ : : ‘ : 
£87 » percentile pupil of the ninth-grade finished the multiple response 


| the modified form in respectively 30 minutes 0 seconds, and 28 min- 
ites 30 seconds. Similar time differences for the 1214 percentile, the 
edian, and the 8714 percentile pupil of each of these three grades are 
ithout exception in favor of the modified form. But with the tenth-, 
+} 


eighth-,.., and the eighth-grade », who took the modified form first, 
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exactly the opposite is true; for these three grades the time diffierences 
shown in Table I are all in favor of the multiple-response. No signifi- 
eant differences in time requirements are shown, moreover, when the 
two hundred six pupils composing all six groups are combined in one 
time-distribution. An advantage is then found in favor of the multiple- 
response in the case of the 121% percentile and the 871% percentile pupil, 
and in favor of the modified form in the ease of the median pupil. Al. 
though somewhat questionable from a statistical standpoint, a computa- 
tion was made of the weighted averages® of the time requirements. The 
advantages thus found are all in favor of the multiple-response, but they 
are slight—only about 6 percent for the 1214 percentile pupil, about 
5 percent for the median pupil, and about 314 percent for the 87% per- 
centile pupil. 

Table II presents evidence to show that the modified form is more 
difficult for the pupils than the conventional multiple-response. While 


TABLE II 
COMPARISON OF THE SCOKES FROM TESTS OF THE MULTIPLE-RESPONSE AND THE 
MopIFIED ForM 


Classes 
Grade Grade Grade Grade Grade Grade 
Vil Vill Ix x VIII est Vil lese 
Number in Class 51 52 47 24 16 16 
Range in Scores 
Multiple-Response 21-55 15-51 19-55 24-56 36-58 22-50 
Modified Form 20-52 16-52 21-55 19-50 27-54 19-47 
Mean of Scores 
Multiple- Response 36.02 39.10 42.11 37.88 46.69 39.19 
Modifhed Form 33.45 36.46 41.66 33.42 38.75 32.19 


8. D. of Scores 


Multiple-Response + 7.38 + 8.47 + 8.87 + 8.28 + 5.45 + 9.10 

Modified Form + 8.38 + 9.48 + 8.46 + 10.38 + 7.00 + 7.91 
8. D. of Means 

Multiple- Response + 1.04 + 1.17 + 1.29 + 1.6 + 1.36 + 2.27 

Modified Form + 1.17 + 1.32 + 1.23 + 2.12 + 1.75 + 1.97 
S. D. of 8. D.’s 

Multiple-Response + 73 + &3 + .92 + 1.20 + 96 + 1.61 

Modified Form 83 t 93 + 87 + 1.50 + 1.24 + 1.40 
M —M. 

Mult.-Response - Modified + 1.65 + 1.50 + 25 + 1.65 + 3.58 + 2.33 
8. D. D. of Means 
8.D —s. D 

Mult.-Response Modified — .90 — 81 + .32 -1.09 — .99 +. 56 


S.D.D.of 8. Ds 








*The weighted averages were computed thus: The median time on the multiple 
response, of, e. g., the 12% percentile pupil of each grade, was multiplied by the 
number of pupils in that grade. The sum of the six products thus obtained was 
divided by the total number of pupils, 206, to give the weighted-average median 
time. Weighted averages were obtained in this way for the 12% percentile, the 
median, and the 87% percentile pupil on both forms. 








mer 
the 

































Oct., 1928] MODIFIED FORM OF MULTIPLE-RESPONSE TEST 215 





. the ranges in scores for all six classes were approximately equal with both 

forms, scored as right and wrong, for example, 15 to 51 points for the 
multiple-response and 16 to 52 points for the modified form with the 
eighth-grade ; 22 to 50 points for the multiple-response and 19 to 47 points 


for the modified form with the eighth-grade,,., etc., the mean scores were 
in every class lower for the modified form than for the conventional 
multiple-response. Thus for the seventh grade, the mean on the multiple- 
response was 36.02 points and for the modified form, 33.45 points; for the 
tenth grade, 37.88 points in the multiple-response and 33.42 points on 
the modified form, ete. When, moreover, the difference of the mean 
scores is divided by the standard deviation of the difference of the 
means,®, *’ the results indicate the degree of significance of the greater 
difficulty of the modified form: Thus, with the eighth-,,,, the eighth-,,,, 
the seventh-, the eighth-, and the tenth-grades, the chances are respec- 
} tively more than 999, about 990, 950, 933, and 950 out of a thousand 
that if the investigation were repeated the mean scores with the modi- 
fied form would again be lower than with the multiple-response. Here, 
then, the greater difficulty of the modified form may be considered 
‘practically certain’’ with the eighth-grade,,,, ‘‘probable’’ with the 
eighth-grade,,., and ‘‘possible’’ with the eighth-, the seventh-, and the 
tenth-grades, 

Table II offers evidence which seems to indicate that the modified form 
liferentiates between the good and the poor pupils at least as well as the 
multiple-response. When the difference of the standard deviations is 
divided by the standard deviation of the differences of the standard 
deviations,* the results show slight values in favor of the modified form 





with four of the groups, and two smaller values in favor of the multiple- 
response. Thus, with the seventh-, the eighth-, the tenth-, and the eighth- 
grade,,,, the chances are respectively about 816, 791, 862, and 839 out of 

thousand, and with the ninth-, and the eighth-grade,,, respectively 
bout 375 and 288 out of a thousand that if the investigation were re- 


S.Di\2 8. D2\2 
S.D.D.x = I( :) 7) 
~” Sire af VN 


"For convenience, this partial formula is here used; the complete formula would 
merely tend to increase, somewhat, and therefore to make somewhat more significant 
the results obtained from this partial formula. 


” 8. D.i\2 8. D2\3 
S.D.D.s.p =. (Fz) oF (Fm) See also footnote 7. 
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peated the differences would again be in favor of the modified form 
Only the results with the tenth-grade, however, can here be considered 
as ‘‘ possible.’ 

Further evidence that the modified form differentiates between the 
good and the poor pupils at least as well as the multiple-response is indi- 
eated in Table II by the greater standard deviations of four of the six 
classes with the modified form than with the multiple-response. 

Table III furnishes a crude estimate of validity which will later be 
checked by means of a more detailed study. An inspection of the mean 
scores made on both forms by certain groupings of pupils in each grade, 
selected on the basis of teacher-judgment without reference to the scores 
made on these tests, indicates that both forms of test are valid, since each 
form differentiates consistently between the best, the medium, and the 
poorest pupils in each class. Thus, in the seventh-grade, on the multiple- 
response the best 20 percent of the pupils averaged 8.31 points higher 


TABLE III 
COMPARISON OF THE MEAN SCORES IN THE MULTIPLE-RESPONSE TEST AND THE 
Moprrrep ForM, oF VARIOUS GROUPS OF PUPILS SELECTED ON THE BASIS OF 
TEACHERS’ JUDGMENTS WITHOUT REGARD TO TEST SCORES 


MBEAN MEAN MEAN 
Scores or Scores or Scores or 
CLASSES Best 20 Percent Mipp.ie 60 Percent Poorest 20 Percenr 
or Pupis or Pupits or Puptts 
Grade VII 
Multiple-Response 43.90 35.59 29.70 
Modified Form 41.80 32.16 28.10 
Difference 2.10 3.43 1.60 
Grade VIII 
Multiple-Response 48.00 40.00 27.30 
Modified Form 47.70 36.38 25.50 
Difference . 30 3.62 1.80 
Grade IX 
Multiple-Respons« 50.44 42.97 31.00 
Modified Form 50.22 42.41 30.67 
Difference .22 . 56 .33 
Grade X 
Multiple-Response 48.00 37.57 28.00 
Modified Form 45.00 33.29 22.20 
Difference 3.00 4.28 5.80 
Grade VIlls1 
Multiple-Response 54.33 46.00 39.33 
Modified Form 48.00 38.00 32.00 
Difference 6.33 8.00 7.33 
Grade VIillese 
Multiple- Response 50. 67 37.10 34.67 
Modified Form 43.00 30.10 28.33 
Difference 7.67 7.00 6.34 


than the middle 60 percent of the pupils, and the middle 60 percent of 
the pupils, 5.89 points higher than the poorest 20 percent of the pupils; 










































No. ¢ MeO» 1928] MODIFIED FORM OF MULTIPLE-RESPONSE TEST 217 


Son the modified form, the best averaged 9.64 points higher than the mid- 
Sdle pupils, and the latter, 4.06 points higher than the poorest pupils. 

= The data in Table III also seem to indicate that the modified form is 
the marclatively at least as valid as the multiple-response, since it differentiates 
Shetween the best, the medium, and the poor pupils at least as well as the 


orm 
pred 


ndi- 

ais itiple-response. With the eighth-grade,,,, the best 20 percent of the 
pupils, selected on the basis of teacher-judgment without reference to 

r be Iaescores On these tests, made a mean score of 7.67 points lower on the modi- 


ean Bfied form than on the multiple-response, the middle 60 percent made a 
mmean score of 7.00 points lower, and the poorest 20 percent a mean score 
56.34 points lower. If with all six classes there had been a similar, pro- 
B cressively smaller difference between the mean scores of these three 
the #egroups on the two tests, the evidence would indicate a consistent superi- 
ple. pority of the multiple-response over the modified form as a means of dif- 
sher Paterentiating between the good and the poor pupils. But the data fail 
to reveal any such consistent superiority of the multiple-response ; in fact 
the results from the tenth-grade indicate the contrary, since the cor- 
sresponding differences are progressively larger instead of smaller. Here 
he differences in the mean scores are for the best 20 percent, 3.00 points; 
*the middle 60 percent, 4.28 points; and for the lowest 20 percent, 
5.80 points. The remaining data in Table III are no less favorable to 
the modified form than to the multiple-response. The differences in 
means on both forms, between the scores of the best 20 percent and the 





middle 60 pereent favor the modified form in the seventh-, eighth-, 


Sninth-, and eighth-grade,,,, since they are progressively larger; while 
:" th these same classes the differences in means on both forms between 
scores of the middle 60 percent and the poorest 20 percent favor 
e multiple-response, since they are progressively smaller. 
The modified form as tested is equally as reliable as the conventional 


iltiple-response. Table IV shows a considerable variation in the re- 


TABLE IV 
COMPARISON OF THE RELIABILITY COEFFICIENTS OF THE TWO HALVES RESPECTIVELY 
OF THE MULTIPLE-RESPONSE TEST AND THE MODIFIED FoRM, COMPUTED BY 
THE PEARSON PrROopUCT-MOMENT FORMULA 


CLASSES 
Grade Grade Grade Grade Grade Grade 
VII Vill IX xX Vil Tas1 VII lesz 


Number in Class 51 52 47 24 16 16 


eliability r 


“7 Multiple-Response 35 78 78 97 $2 84 
Lis Modified Form 57 68 . 87 93 49 7 
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liability coefficients of the two forms when correlations are computed 
between the scores on the two halves of the tests. When the reliability 
coefficient is computed for all six classes combined as one group, hoy. 


ever, the coefficient is found to be .70 for the multiple-response and 7} 
for the modified form, a difference of .03 in favor of the modified form 
But the relatively small number of cases and classes used in this inves. 
tigation, and the size of the probable error as compared with the dif. 
ference, prevent definite conclusions regarding the relative reliabilities 
of the two forms. 

Table V indicates that the reliability of the modified form as com 
puted by Brown’s formula from these tests in science, is nearly as high 


TABLE V 
COMPARISON OF THE RELIABILITIES OF ONE HUNDRED ITEMS OF VARIOUS TYPES wf 
TESTS REPORTED BY RUCH, AND BY MCCLUSKY AND CURTIS, WITH RE- 
LIABILITIES OF ONE HUNDRED ITEMS OF THE AUTHORS’ MULTIPLE- 
RESPONSE TEST AND Mopiriep Form Test, COMPUTED 
BY Brown’s FoRMULA* 





Tree anp Treat NUMBER OF RELIABILITY r, RELIABILITY r, 
Pupi.s Har versus Hacrt 100 Irems 
Ruch's 
Recall 562 81 + .010 .90 
Five-response 137 .80 + .020 . 89 
Three-response 134 .60 + .037 75 
Two-response 135 .74 + .027 . 85 
True-False 133 .56 + .040 .71 
MeClusky-Curtis 
True-false (science) 115 .53 + .045 . 7643 
Modified Form (science) 115 .76 + .026 9043 
Authors’ 
Five-response 206 .70 + .024 . 859t 
Modified Form 206 .73 + .022 . 8512 
ar 
t% = a 
1+i(n-—- lr 


tFifty items for each of Ruch’'s tests; twenty-five items each for McClusky and Curtis’ tests; asi 
sixty items each for the authors’ tests 

{Corrected for dispersion, and thus made comparable to these r’s reported by Ruch, by means of tht 
To VTua(i—R1) 
Loo i} Rai -T yu) 
Co., 1923), p. 221 


formula Kelley, Truman L.- Statistical Method (New York: The Macmillz 


as the reliability of the recall and multiple-response tests experimente! 
upon by Ruch,® and as the modification of the true-false experimente 
upon by McClusky and Curtis.”° The reliabilities of Ruch’s recall an¢ 


* Ruch, Giles M. The Improvement of the Written Examination (New York 


Scott Foresman & Co., 1925), p. 111. 
* McClusky, Howard Y. and Curtis, Francis D. ‘‘A Modified Form of the True 


False Test,’’ Journal of Educational Research, XTV (1926), 213-224. 
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five-response tests were respectively .90 and .89; of the modified form 
of the true-false (science) .90; of the authors’ five-response tests, .86, 
and of the modified form of the multiple-respense, .85. 

Several other statements seem justified by the authors’ extensive use 
of both these forms, although no objective evidence supporting them 
ean here be presented. It apparently takes no longer to construct test 
items of the modified form than of the multiple-response; also, pupils 
are able to score this form readily, though they, as well as the examiner, 
require a little more time to score the modified form than the multiple- 
response. The modified form seems, moreover, to develop in the pupils 
to a greater extent than the conventional type, the habit of critical 
scrutiny and analysis of every item; it tends, furthermore, to eliminate 
whatever elements of chance may lurk in the multiple-response. 

An examination of individual test papers indicates, too, that the 
modified form is superior to the multiple-response for diagnosis of in- 
dividual and class difficulties and weaknesses. 


SUMMARY 

In so far as the results of this investigation may be indicative, it 
seems reasonable to conclude that this modified form of the multiple- 
esponse test is more difficult than the conventional multiple-response. 
[t is inferior in that it requires somewhat more time to correct and score, 
though it requires very little, if any, longer than the conventional form 
to administer. It is at least as good in the following respects: (1) It 
takes no longer to construct. (2) It differentiates at least as well be 
tween the good and the poor pupils. (3) It is probably as valid. (4) 
It is at least as reliable as the conventional form. It is superior to the 
conventional form in the following respects: (1) It possesses greater 
value for diagnosis of individual and class difficulties and weaknesses. 


2) It stimulates more thinking on the part of the pupil while he is tak 
ng the test. (3) It tends to eliminate whatever guessing factor may 


operate in the multiple-response. 





UNFAMILIAR SITUATIONS AS A DIFFICULTY IN SOLVING 
ARITHMETIC PROBLEMS 


CARLETON WASHBURNE 
Superintendent of Schools 


and 
MABEL VOGEL MORPHETT 


Director of Research Department, Winnetka, Illinois 

Can children solve arithmetic problems presenting situations within 
their own experience more readily than they can solve problems involving 
unfamiliar situations? In other words, all other elements being equal, 
is the situation a real factor in the difficulty of an arithmetie problem? 

To answer this question a problem solving test made up of eight pairs 
of problems was devised, at the fifth-grade level. The two problems that 
made up a pair involved the same arithmetic processes, numbers of the 
same order of difficulty, and the same sentence structure and length; they 
differed only in the situation involved (see Table I). Six of the pairs 
consisted of two-step problems, i.e., problems requiring two different 
arithmetic processes; the remaining two pairs required only one step 
for their solution. The arithmetic processes involved were: column ad- 
dition; subtraction with carrying; simple multiplication; short divi- 
sion; compound multiplication; and long division. 

The two members of each pair were separated and the sixteen prob- 
lems mixed and presented in random order. 

The test was mimeographed and given to 441 children in the fifth 
grades of the publie schools of Wilmette, Batavia, Dolton, Highland 
Park, and Winnetka (all in Illinois) and of the Francis W. Parker 
School (Chicago). 

The tests were scored on the basis of process and accuracy. The 
process was counted correct if the child showed that he knew how to 
work the problem, but had made some error in caleulation. A child had 
to have the eorreet answer in order to be marked correct in accuracy. 

Table I gives the results of the test, for both process and accuracy. 
Comparable problems are shown, together with the percent correct in 
process and accuracy for both the familiar and unfamiliar situations. 

Let us take the first two examples on the table. The processes used 
are column addition and simple multiplication. The wording of the two 
problems is nearly identical. The number difficulty of the two problems 

‘This study was carried out by the Winnetka Research Department to supplement 


the studies of the Committee of Seven reported by Carleton Washburne and Raymond 
Osborne in the Elementary School Journal for November, 1926. 
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TABLE I 
PROBLEM SOLVING TEST 
FirtH GRADE 


PAIR 1 
UNFAMILIAR ADDITION — MULTIPLICATION 
4 steam shovel made an excavation for a building. It dug out 26 truck-loads on Monday, 
43 on Tuesday, and 56 on Wednesday. Each truck-load weighed 3 tons. The owner of the 
shovel is going to charge his employers for the work. How many tons will he have to charge 


FAMILIAR ADDITION = MULTIPLICATION 
Some of the children are going to make booklets for a bazaar. Miss Graham's room is 
going to make 85 booklets; Miss Leap’s room, 42 booklets; and Miss Damberg’s, 46 booklets 
e will be 4 sheets in each booklet. John is going to buy the paper for all the rooms 
How many sheets will he have to buy for them? 


PERCENT CORRECT 


Process Accuracy 
Unfamiliar 51 45 
Familiar 67 60 

PAIR 2 


UNFAMILIAR ADDITION = DIVISION 
In August a farmer used 64 bushels of prepared feed; in September, 72 bushels; in 
October, 73 bushels; and in November, 63 He has 8 cows. How much prepared feed did 
ach cow eat? 
FAMILIAR ADDITION = DIVISION 
On Monday we took in 66 pins at our circus; on Tuesday, 96 pins; on Wednesday 
Thursday, 61. There are 8 of us in the circus. How many pins does each of us 


PERCENT CORRECT 


Proces Ac racy 
Unfamiliar 73 58 
Familiar 75 60 

PAIR 8 


UNFAMILIAR MULTIPLICATION — SUBTRACTION 
A contractor needed 820 bolts in order to complete a building job. Hi ‘ > bolt 
each of 830 packages. How many more bolts does he need to complete the job? 


FAMILIAR MULTIPLICATION — SUBTRACTION 
Jack needed to sell 920 copies of Liberties in order to win a baseball glove He sold 
44 Liberties each week for 20 weeks. How many more copies does he have to sell to win the 
: vel 
PERCENT CORRECT 


Process Accuracy 
Unfamiliar 66 61 
Familiar 65 62 
PAIR 4 
UNFAMILIAR ADDITION — SUBTRACTION 
A telephone company had 62 spools of wire Yesterday it received 11 from a nearby; 
ctory. Today it received 39 more from there. Then 15 of them w ed. How many 
spools of wire should the company have now! 


FAMILIAR ADDITION — SUBTRACTION 
I had 55 marbles. Yesterday my daddy gave me 23 new ones Today my brother 
gave me 35 of his. Then I gave my chum 15 of them. I wonder if I have lost any How 
many marbles should I have now? 


PERCENT CORRECT 
Process Accuracy 
Unfamiliar 74 63 
Familiar 85 71 
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PAIR 5 
UNFAMILIAR MULTIPLICATION 


A merchant sold 20 bags of charcoal. Each bag held 35 pieces. How many pieces of 
charcoal did he sell? 


FAMILIAR MULTIPLICATION 


The girls have to make 30 boxes of taffy. Each of the boxes holds 25 pieces. How 
many pieces of taffy do they have to make! 


PERCENT CORRECT 


Process Accuracy 
Unfamiliar 91 79 
Familiar 88 79 
PAIR 6 
UNFAMILIAR DIVISION = MULTIPLICATION 
3 c typist wrote some letters for a lawyer She typed a letter of 357 words i 
She wants to know how long a letter she could write if she typed for 9 minutes 


FAMILIAR DIVISION = MULTIPLICATION 
hn and his father took a trip to Canada. One day they drove 147 miles in 7 hours 


John wants to know how far they could go if they drove 9 hours the next day. 
PERCENT CORRECT 
Process Accuracy \ 
Unfamiliar 44 41 
Familiar 43 40 
PAIR 7 


UNFAMILIAR DIVISION 
19 chemists bought 627 Florentine flasks Each chemist is to have the same number 
flasks. How many flasks should each chemist have! 


FAMILIAR DIVISION 


18 Boy Scouts caught 396 fish while on a fishing trip Each scout is to have the sar 
number of fish How many fish should each scout take home? 


PERCENT CORRECT ' 
Process Accuracy A 

Unfamiliar 82 65 

Familiar 91 81 
| 
' 

PAIR 8 
UNFAMILIAR SUBTRACTION — DIVISION 
A coke plant had to turn out 673 tons of coke for a steel company It has already 
produced 129 tons There are 32 ovens in the plant. How many more tons will each ove! 

have to produce? 


FAMILIAR SUBTRACTION — DIVISION 





My committee has been asked to dress 363 pencils for the bazaar. We have already dresse 
149 pencils. There are 23 girls on the committee. How many more pencils will each girl hav 
te dress! 
PERCENT CORRECT 
Process Accuracy 
Unfamiliar 53 45 
Familiar 7 


ene en: 


71 54 


is exactly comparable. The only variable that seems to affect the solu- 
tion is the familiarity of the situation. The unfamiliar problem invol\ 
ing truck loads of dirt for a building excavation was correctly attacked 
by 51 percent of the children and was correctly solved by 45 percent of 
the children. The familiar problem involving a more childlike situation 


ee a a 





Uc 
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in a classroom was correctly attacked by 67 percent of the children and 
was correctly solved by 60 percent of the children. The problem in- 
volving a familiar, childlike situation is attacked and solved correetly by 
© a larger percentage of children. 

ow é In pairs 1, 4, 7, and 8 a decidedly larger percentage of children was 
able to solve the problems containing familiar situations than were 
able to solve problems containing unfamiliar situations. In pairs 2, 3, 
5, and 6, there is no significant difference between the scores of the two 
sets of problems. 

In three cases the failure to get a significant difference may lie with 
™ the test. Pair 2, for example, compares feeding cows with collecting 
pins for a cireus. The former, intended to be unfamiliar, is probably 
fully as familiar as the latter. The difference between selling bags of 








chareoal and filling bags with taffy (pair 5) is probably not significant. 


Typing letters (pair 6) seems a priori less familiar than taking auto 

rides, but may not really be. But certainly pair 3 should present a con- 
; trast—selling Liberties must be more familiar than completing a build- 
® ing with bolts; it is diffieult to understand why both of these situations 
‘aused the same amount of difficulty. 
: Notwithstanding these exceptions, the fact remains, however, that 
= whenever a significant difference occurs in the percentage of children 
solving correctly the two sets of problems, it is always the problem in- 


oe 


volving the more familiar, childlike situation which gets the higher score. 


_— 


: Table II summarizes the data. This table shows that where one form 
(= or the other of a problem is missed, there is just about twice as good a 

hanee (429 to 218) of the child missing the form involving an un- 
familiar situation as there is of his missing the one where the situation 
is familiar. 








TABLE II 
q 
THE NUMBER OF RIGHTS AND WRONGS FOR EACH PAIR OF PROBLEMS 
UNFAMILIAR UNFAMILIAR UNFAMILIAR UNFAMILIAR 
| RIGHT WRONG Rigut Wrone . 
OBLEM Fawmar Rient Famiuiar Wrone Faminian Wrono Fawr Ricnr TOTA! 
197 109 35 100 441 
289 79 32 41 441 
> 247 105 458 41 441 
: ‘ 298 40 27 76 441 
369 23 31 18 441 
181 232 16 12 441 
7 351 31 4 50 441 
8 221 109 20 91 441 


Totals 2153 728 is 429 3528 
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The data presented herein bear out the common sense conclusion, 
and the preliminary experiments of the Committee of Seven, in show. 
ing that part of the difficulty that children have in applying their arith. 
metic to textbook problems lies in the problem, not the children. Text 
book makers and teachers alike should make the problems they expect 
children to solve, childlike and real. The three following criteria, al! 


of which were applied in the making of the ‘‘familiar’’ problems of this J 


experiment, should prove helpful in problem making: 
1. The situation should be familiar and childlike—i.e., one in whieh 
the child can easily imagine himself as being. 





2. The principal character in each problem should have a reason for | 


solving it—i.e., if the child were in the situation described in the prob- 
lem, he would feel a real need for finding the solution for which the 


problem ealls. 

3. The vocabulary and sentence structure of the problem should, of 
course, be within the reading ability of the child who is expected to 
solve it. 














» A TABLE FOR PREDICTING THE VALIDITY AND RELIABIL- 
? ITY COEFFICIENTS OF A TEST WHEN LENGTHENED oI 
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KEEPING constant the factors of difficulty and type of content, an in- : 


ert 


PO 


crease in the length of a test may be expected to raise both its validity 
and reliability coefficients. 

In test construction, for example, the question often arises, ‘‘ Shall - 
™ | increase the length of the present test in order to increase the validity 
m@ and reliability of the predictions, or shall I try a different test, that is, 


oS 
rs 


this 


for one made up of a different content?’’ The accompanying table will help 

ob. to answer this problem. For, with a knowledge of the validity and re- MI 

the | liability coefficients to be expected in a lengthened test of feasible length, i 
ne may adopt an arbitrary rule, such as the following, to guide his 

of procedure :—‘If the addition of one minute of test time will not add 

to 005 to the reliability coefficient or .002 to the validity coefficient, then 


t is not worth while to increase the length of the present test further. , 
® and one should search then instead for different test content.’ 





The new items, when added, should parallel the present items in difficulty and ' 
should be ‘‘sandwiched in’’ at the proper difficulty levels. If t is the increase in the 
time limit of the present test, the increase per minute in either validity or reliability 
is 1/t times the actual increase in validity or reliability respectively. If a three- 
minute test has an initial reliability of .60, then doubling the test leads us to expect | 
a reliability of the lengthened test of .750; tripling, .818; quadrupling, .857; 
quintupling, .882; sextupling, .900; and septupling, .913. Since the increase at 

ch step in time is 3 minutes, t= 3, and it follows that 1/t— %. One-third of ; 
750 — .60) is more than .005; so also is % (.818 — .750); and ™% (.857 — .818); 
ly (.882 — .857); and % (.900 — .882); but not 4 (.913 — .900) which is only .004. 
Therefore, as far as reliability is concerned, we may say that it is not particularly 
rofitable to make the lengthened test contain more than six times the present number 
f items. If the test originally contained 50 items, the lengthened test would 
= now contain 300 items. For each item of the original test we need to supply five 
» other items of the same difficulty and kind of test content, or 250 more questions. 
The number of multiples, n, evidently refers then to an increase in the time limit 
as well as to an increase in the number of questions. If the original test rapidly 
becomes so difficult that no test subject attempted the last five items, say, it may 
suffice merely to add five test items of parallel difficulty to each of the first forty-five 
questions and to sextuple the time limit on the test. Time required in other opera- 
® tions than working on the test content,—time for reading directions, for example 
= should be excluded from t in the above computations involving the expression, 1/t. 
» The final test, then, would consist of 300 items; would require eighteen minutes; 
and would have a reliability of .90. In general it would cost six times as much to 
print such a test; six times as much to score it, and so on in like proportion. This 
cost consideration might lead us to abandon the test even though the reliability is 
now acceptable. The more pertinent consideration is always validity rather than 
reliability, for no test, however reliable, if used by itself, is of any worth unless 
its validity is acceptable. 


I ) 
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The accompanying table will show by inspection all the various in 
terrelationships of length of test, validity coefficients and reliability 
coefficients between certain wide limits. The table gives the expected 
validity and reliability coefficients, as dependent upon (a) the present 
reliability, rap, and upon (b) the increased number of questions for 
lengthened tests which would contain twice the present number of ques- 
tions (n= 2), three times the present number of questions (n ==3), up 
to and inclusive of a test which would contain fifteen times the present 
number of questions (n==15). This more than covers all commonly 
needed cases. 

USE OF THE TABLE 

Suppose we have a three-minute test which has a reliability coefficient 
(ran) of .60, and which correlates .40 with a criterion of, say, scholar- 
ship. The validity coefficient upon lengthening is given by the follow 
ing formula, (A) :—1r (na) K.ry,. In which, ry; (a4) is the validity co 
efficient of the lengthened test; K is the plain-faced number of the table, 
corresponding to the given initial reliability coefficient, raz, and the num 
ber of multiples, n, contained in the longer test; r,;, is the initial validity 


coefficient of the test. In the above example, where, ri, == .40, and 
Tap .60 by following horizontally across the page in the row, rap = .60, 
we may note the following successive values of K (plain-faced numbers, 
always larger than 1.00) as dependent upon n:—When ry, — .60, and 
n 2 &==1.18: n 3, K 1.168; n 4, K 1.195: n d, 
KK 1.213; ete. This means, by formula (A) above, that if we double 


the length of the test, we may expect the thus lengthened test to cor 
relate with scholarship to the extent of 1.118 times the present amount, 
i.e., 1.118 (.40) —.45; with the test tripled in length, 1.168 (.40) AT; 
with the test quadrupled in length, 1.195 (.40) 48; and with the test 
quintupled in length, 1.213 (.40) 485, ete., ete. The above series shows 
that little additional validity is to be gained after this test has been 
tripled or quadrupled in length. If the test were tripled in length, 
number of questions, and time limit, so that it were then a nine-minute 
test, the test would have a validity coefficient of .47; and the reliability 
of this nine-minute test would be .818. 

This reliability coefficient is an italicized entry found in the table in 
the 60-row and in the column, n=3 just beneath the above found K 
entry of 1.168. The value of the increased reliability coefficient is thus 
found directly and without any computation whatsoever. These italicized 


figures are always less than 1.00. 
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- We may be interested to know, in the above example, to what extent } 
7 we may have to increase the length of the test in order to secure a re- ' 
ed § bility coefficient of, say, .90. Tracing the reliability coefficients (itali- 
- eized figures) in the ray .60 row, to the right until we find the first 
sai entry as large as .90, we find the entry .900, and directly beneath it at 
“* E® the bottom edge of the chart (or at the head of the same column) the 
fe’ desired answer, n==6. If our desired reliability is .90, we shall have to ak 
- make the three-minute test into an eighteen-minute test. If we wish to { 
“YT have a reliability coefficient of .95, then the test must be made to contain 
| thirteen times the present number of items. 
i} We may also have in mind a desired, or ‘‘ideal’’ or minimally ace- 
mt ceptable validity coefficient. If the above results as to reliability and 
= idity were obtained in a thirty-minute university intelligence test, 
. we might not feel satisfied with a validity less than .50, perhaps. Sub- 
” stituting the desired validity coefficient .50, for r; (n4), and 40 for rya, ; 
sia 1 n formula (A), we obtain, .50—K(.40). Solving for K, we obtain, / 
“ K— 1.25. If, now, we trace across to the right in the bold-faced num- 4 


bers of row .60, we find the entry, 1.250 corresponds exactly to n 10. 
That is, this half-hour university intelligence test would need to be in- 
yU ~ 


eased to 10 times 30 or 300 minutes, or 5 hours, in length in order to 





. tain a validity coefficient of .50. Evidently, the improvement is not 

" vorth the cost in time. Although the five hour test has a high reliabil- 

le 938), the validity is still low. Five hours of test time in this ease 

ean be far better spent on different content. 

t, | The validity coefficient improves much more slowly than the reliabil- 

7 ty coefficient as the result of lengthening the test. By looking in the 

st column headed x we note that if the above half-hour university intel- 

vs ligence test were made infinitely long it would have a validity only 1.291 
n times the present validity coefficient, or 1.291 (.40) 164. If five 


, hours of testing give us a validity of .50 and if an infinite number of 
hours of testing give us .516, then obviously it is not worth the while, 

\ ia ' ° 3° . . 

: \j so far as validity is concerned to make the test much longer than five 


i hours in length. The value of the reliability coefficient for any test when 





made infinitely long is always 1.00. The definition of a ‘‘true score” is 
\ Sy ° . ° . ° } 

} thus determined; a true score is any seore which is perfectly reliable, 
1s i . « — ess 
or it is the average score made on an infinite number of repetitions of 
u 


a test. 
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THE FORMULAS 
Lack of space prohibits us from deriving the equations used. 
Formula (B) for the increased reliability coefficient is :— 
T-Tas 
P(aa)(m) =F (a —1) ras. 
The formula for the increased validity coefficient is adequately de- 
fined by substituting in Formula (A) the following formula (C) for K: 








” n n 


Vn+n(n—1) i_ wa + (n—1) ras 
It is desirable to keep in mind the assumptions involved in the develop- 
ment of Formulas (A) and (B). Formula (A), or (C), assumes: 


1. That the correlation of the first form of the test with the criterion, 
(ry) is of the same numerical value as the average correlation 
of all other multiples with the same criterion. This is a fair as- 
sumption probably, at least for the length of test time involved 
for most test scales,—a length of time probably ordinarily quite 
too short for practice effects to play too great a part. Inasmuch as 
one ordinarily cannot obtain a reliability coefficient without hav- 
ing two alternative forms of the test prepared, thus making it 
possible to correlate both forms of the test with the criterion, it 
is better to take for r,, of formula (A) the average of the two 
validity coefficients; and if more than two are available, the aver- 
age of all of them. 

2. That the standard deviations of the successive forms, or multiples 
of the original tests are all equal. If the new questions are equally 
as well scaled as the original, then this is also probably a fair 
assumption. 

3. That the correlation, rap, the reliability coefficient obtained from 
the first two forms of the test, is of the same numerical value as the 
average of all such possible intercorrelation reliability coefficients. 
If practice does not affect the results within the test time limit of 
the total test, then this is a fair assumption. As in assumption 1 
above, one should use for ra, in both formulas (A) and (B) the 
average of any and all reliability coefficients of the different forms 
he has at hand, or can readily obtain. It can readily be seen that, 
with n alternative forms of a test given, one of our pressing needs 
is a formula for telling us what is the average Pearson intercorrela- 

n (n—1) 

2 


intereorrelation coefficients. This formula is now available.? 





_ 


tion coefficient of the n forms, without solving all the 


*Edgerton, H. A. and Toops, H. A. ‘‘A Formula for Finding the Average 
Intercorrelation Coefficient of Unranked Scores Without Solving Any of the In- 
dividual Intercorrelations.’’ Journal of Educational Psychology. January, 1928 
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q In formula (B) above, assumptions 2 and 3 are both made, and no 
others. 

It will be noted that, the lower the initial reliability of a test, the 
greater is the hope of improving it, both in reliability and validity, by 
increasing the number of test elements. We also see that a test of low 
initial validity is seldom worth attempting to salvage by increasing its 
length. 

The limits of improvement in validity are found by taking limits, 
as n approaches infinity, in formula (C). This yields formula (D): 





: ; a sia . Tra a 
Maximum possible validity =r, (« x) = ——=—. This is to be defined 
V Tas 
as the correlation of a criterion and a true test score, or the correlation 
of an infinite number of forms of the test with the eriterion. Formula 


D) yields the K-values of the column, (n 2), of the table. 





Taking limits similarly in formula (B) we obtain r ; : 

: (co X) (oX 

1.00 as the result. True test scores are thus perfectly reliable; they 

4 yield a perfect measure of the test subject’s ability on fhat test, but do 

; not, of course, correlate perfectly with any criterion. Thus it follows 

' that we may have a test of perfect reliability (r43, 1.00) but of zero 

validity (m4==0); the reverse however, is never possible. The num- 

er of hairs on one’s head counted on successive days, might be expected 

to have about perfect reliability but when used as a test of scholarship, 
might be expected not to correlate with scholarship at all! 


CONCLUSIONS 


} A table has been constructed which will enable one to determine by 
inspection, and with no computation beyond one simple multiplication, 
the increased validity and reliability of a test when lengthened to any 

» desired predetermined limit, so long as the test is not lengthened to more 

= than fifteen times its present length. By reversing the formulas, it is 

readily possible to determine how many multiples of the present length 
of test are required respectively to secure any predetermined reliabil- 
ity coefficient, say, .80 or .85 or .90 or any predetermined, obtainable, 
validity coefficient. The limits of improvability both as to validity and 
reliability are also shown. The table may be used not only with tests, 
but is also useful in experimental work generally to determine the num- 
ber of repetitions necessary to produce a desired validity or reliability. 

Thus, if two examinations (medical, physical, civil service examinations, 

school examinations, or what not) yield marks which correlate with each 
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other to the extent of .65, the examinations are too short,—contain too 
few questions or scoring points or have too short a time limit; the table 
shows that a minimally acceptable reliability of, say, .85 would require 
in this ease the examinations to contain three times as many questions, 
or items. In test construction, for example, tests of low initial reliabil- 
ity and fair validity may often be salvaged by the simple expedient of 
making them longer. The table makes this decision a very easy matter. 
Many of the published tables of intercorrelation coefficients are value- 
less for the reason that the tests used were so short that a repetition of 
the experiment might yield almost any other values than those published. 
The authors propose, for purposes of discussion, that no intercorrelation 
tables should ordinarily be presented unless based on tests or units 
(those on which the intercorrelations are based), which have at least 
a reliability coefficient of .90, unless the reliability coefficients are given, 
or unless the nature of the experiment precludes the possibility of such 
reliability being attained. Such a standard, if used in the past would 
have saved, for example, our being assailed in the literature by state- 
ments in regard to ‘‘the correlation between mechanical ability and in- 
telligence’’ ranging from, say,—.60 to-+ .60. Such a consideration, 
again, for example, would require, in general, that we repeat the ma 
jority of psychomotor tests many times and average the individual’s 


several seores in order to obtain a reliable seore for the individual. 

















A COMPARISON OF TWO METHODS OF LEARNING 


R. D. RUSSELL 
University of Idaho 


Ir has been said that there are four methods of approach to learning: 
1. Studying the thing itself; 
2. Studying a picture of the thing; 
3. Hearing about it; 
4. Reading about it. 
Within the last few years Freeman, McCluskey, James, Weber and 
others have made extended investigations regarding the value of pictures 
in teaching. Only incidentally have they considered the relative effi- 
ciency of learning by lecture and reading the material. The writer is not 
informed of any comparisons having been made between learning by 
reading and by hearing material read. 
A number of investigators have attempted to determine whether 
the visual or auditory method of presentation gives a better result i 


the recall of the material presented. Hawkins’ presented words visually 

and orally. The object of the experiment was to determine which was 

e stronger, auditory or visual memory. His results show that in the 

primary and grammar schools students recall more from auditorially pre- 

nted material, while in the high school and college the visual method 
an advantage. 

Whitehead? presented nonsense syllables visually and orally to 
dults. He found the visual method superior in memorizing but ma 
erial memorized orally was retained slightly better. Calkins* experi 
mented with college students. Words were presented visually and orally. 
She found a superiority in favor of the visual method. 

Pohlmann* studied the effect of visual, auditory, visual-auditory and 
visual-auditory-motor presentations of objects. words, non-sense syllables 
nd numbers, on school children from 9 to 14 years of age. He found 

at auditory presentation is better than visual with significant material, 
but that visual presentation is better with nonsense material. 

The studies referred to above make use of nonsense material, num 
bers, and words. The present study differs from them in that it makes 
ise of context material. Several studies have been made, however, of 

ming by lecture and by reading. 

‘Hawkins, Chauncey J. ‘‘Experiments on Memory Types.’’ Psychological 
Review. (Vol. 4, 1897), pp. 289-294. 

* Whitehead, Louis Grant. ‘‘A Study of Visual and Aural Memory Processes.’’ 
Psychological Review. (Vol. 3, 1896), pp. 259-269. ; 

*Calkins, Mary Whiton. ‘‘Short Studies in Memory and in Association from 
Wellesley College.’’ Psychological Review. (Vol. 5, 1898), pp. 451-462. 
*Pohimann, A. Experimentelle Beitrage zur Lehre vom Gedachtnis. 1905 
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Lacy® made a study in which he found the oral presentation superior 
to the visual. His subjects were 7th, 8th and 9th grade children. 

Weber® has recently made a study of the relative effectiveness of 
four methods of presentation, namely, study, teacher, film and fiim- 
lecture. He compared the relative effect of the lecture by the teacher 
to the students with study by the pupils themselves. The students who 
read the lesson for themselves made on the average of .6 of a point 
better than the students to whom the teacher lectured. The experiment 
was with 7th grade students. 

James’ has recently made a very comprehensive study involving six 
different methods of presentation. He found in the 8th grade a su- 
periority in favor of reading. 

This study differs from those of Lacy, Weber and James in that 
the material was read to the students instead of given in the form of 
a lecture. It is hoped that this study will give some suggestion as to 
the most economical period for the introduction of book instruction in 
the elementary school. Johnson* states regarding history that: 

‘*Below the 7th grade the ability of the average pupil to comprehend is so far 
in advance of his ability to read that the necessary details are most economically 
and efficiently supplied by the teacher. Beginning with the 7th grade required 
readings outside of the text may, to an ever increasing extent, be substituted by 
contributions of the teacher.’’ 

The material used in this investigation was a composition of about 
1000 words on the Mongoose. The material was used concerning this 
animal because it, in all probability, would not duplicate the work of 
the children in school, and further because it represented a type of 


material common to schoolroom use. 


METHOD OF INVESTIGATION 

The National Intelligence Test was given to 1080 pupils in the 5th, 
7th and 9th grades. The students in each grade were then divided into 
three groups of equal ability as measured by this test. Due to a loss 
in grouping this number was reduced to 690. 

The. material was presented by the regular class-room teacher in 
three different ways. In one case it was read to the students twice in 

*Lacy, John V. ‘‘The Relative Value of Motion Pictures as an Educational 


Ageney.’’ Teachers College Record. (Vol. 20, November, 1919), pp. 453-465. 
* Weber, Joseph J. Comparative Effectiveness of Some Visual Aids in Seventh 


Grade Instruction. (The Educational Screen, Inc., 1922), p. 90. 
James, Hadden W. ‘‘Relative Effectiveness of Six Forms of Lesson pres 
itation: Film, Leeture, Still-Pieture, Film-Picture, Film-Music, and Reading with 


Particular Emphasis on the Efficiency of Different Types of Material for Film 
Presentation.’’ An unpublished thesis in the Department of Education, State Uni 
versity of Iowa, 1923, page 241. 

* Johnson, Henry. Teaching of History. 1915, page 287. 
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suecession. The second procedure was to give the students the material 
and ask them to read it at their normal rate, giving them the same 
amount of time for reading as the teacher took in the first method in 
eading it to them. The third group read the material keeping the 
number of impressions constant with that of the first group without 

gard to time. This group was simply told to read the material through 
twice at their normal rate and as soon as they finished to stop and raise 
their hands. 

After the presentation of the material the pupils were given two 
tests, one of the essay type and the other the true-false, the essay type 
being administered first. The scores on these tests were combined on the 
basis of the ratio of their true standard deviations to facilitate com- 
parison of groups. After pairing the scores in Grade 5 only 72 pupils 
remained in each of the three groups. Following are the averages of the 


combined scores: 


GRADE 5 
TEACHERS READING PUPILS READING PUPILS READING 
Tims CONSTANT IMPRESSIONS 
CONSTANT 
(1) (2) (3) 
Means 57.29 49.58 53.95 
S.D.'s 16.329 14.754 17.079 
S.D.M.’s 1.926 1.74 2.016 
8.D m1-Ms 2.595 
S.D mi-M2 2.788 


In comparing the scores made when the teacher read to the pupils 
nd when the pupils read the material for themselves in the length of 
time required of the teacher to read, it will be observed that the mean 
of column 1 is 7.71 points higher than that of column 2. This difference 
7.71) is 2.97 times the standard deviation of the difference which means 
at the chances are 332 to 1 that the difference represents a real supe- 
ority of group one over group two and is not to be aeeounted for by 
e mere operation of chance. 
A comparison of columns one and three shows also a distinct supe- 
rity in favor of the method of reading the material to the pupils. 
After grouping 104 students remained in each of the three groups 


‘f Grade 7. The means for these groups are as follows: 


GRADE 7 
TEACHERS READING PUPILS READING PuPILs READING 
TIME CONSTANT IMPRESSIONS 
CONSTANT 
(1) (2) (3) 
Means 75.79 76.04 75.43 
8.D."s 17.187 16.539 17.844 
8 > M.'s 1.685 1.622 1.749 
8.D 
ms-me 2.328 
8D 2.428 


™Mi-M: 
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It will be observed that the differences in the means of columns 1, 
2 and 3 are negligible in Grade 7. They are no greater than chance 
would allow. However, the students represented by column 2 read the 
material 2.4 times. Although the time was constant and the results 
practically the same it may be possible that the group having the greate: 
number of impressions has a slight advantage. 

In comparing columns 1 and 3 it will be noticed that no significant 
difference is apparent. The students represented in column 3 read the 
material over twice in a slightly shorter period of time than the teacher 
took in reading to the students represented in column 1. 

Three groups of 51 students each were represented in Grade 9. The} 


results follow: 


GRADE 9 


TEACHERS READING PUPILS READING PUPILS READING | 
Time CONSTANT IMPRESSIONS 
CONSTANT 
(1) (2) (3) 

Means 85.41 86.77 87.88 
8.D.'s 12.576 12.999 15.087 j 
8.D.M.'s 1.761 1.821 2.113 
S.D mi-Ms 2.5383 
=D My-My 2.751 


A comparison of columns 1 and 2 of Grade 9 shows a difference of 
1.36 points in favor of the students reading the material themselves 
This difference, however, is not significant in as much as the Standard 
Deviation of the difference is 2.5. A record of the number of times th y 
students read the material in column 1 was not kept. 

Column 3, in which the students read the material twice to compare 
with the number of impressions of those in column 1, shows a superiority 


a 


over column 1. 

The result of the experiment in Grade 9 indicates possibly that stu 
dents learn more from reading themselves than from hearing someone } 
else read when the time and the number of impressions are held constant 


CONCLUSIONS 
The result of this investigation seems to justify the conclusions that: } 
1. Students learn more from having material read to them than in} 
reading it themselves in Grade 5. 
2. The relative effectiveness of the two methods is practically equa 
in Grade 7. 
3. The advantage is slightly in favor of the method of learning by} 





reading in Grade 9. 











DIN 
xs 





ont 


ant. j 


a ©& Ss ”™ 
* $e 
a ie a a ee ee 


esate 





ee ee 





AN EXPERIMENTAL INVESTIGATION OF THE RELATIVE 
EFFECTIVENESS OF TWO PLANS OF SUPERVISED STUDY 
HakL R. DovuGLass 
Unwerstty of Pennsylvania 


Tue chief purpose of the group of experiments described in the fol- 
wing paragraphs was to determine the relative efficiency of the S-R 
sequence and the R-S sequence in the administration of supervised study. 
By ‘‘S-R sequence’’ is meant nothing more than having supervised study 
precede the recitation in the class period and by ‘‘R-S sequence’’ that the 
recitation precedes the supervised study. 

Many progressive teachers employing supervised study do not attempt 
to divide each class period into two portions—supervised study and recita- 
tion—but prefer to plan in larger than daily units covering a number 
of days or even weeks, e. g., challenges, projects, contracts, ete., and the 
majority of those nationally eminent in the field of philosophy and 
practice of teaching lean heavily in the direction of these larger units. 
Yet it is quite likely that in the majority of classrooms where some form 

f supervised study is used regularly and in which the lengthened period 

is available, daily or almost daily recitations are in vogue. The fact 

that this study is based on the latter practice should not be taken as an 
ndieation that the author does not favor some form of larger units for 
ose teachers who are able to plan them. 

Past practice has very largely favored the R-S sequence—study fol- 
owing recitation. It has been advanced that the recitation should begin 
vhere the diseussion left off and should follow assignment as closely as 
ossible. It has been claimed that the study of material to be recited 
mmediately after is not as likely to result in permanent learning as study 


‘ 


with a view to retention for at least 231% hours. In connection with this 
claim, it is offered, particularly by those who think of teaching as lesson 
earning and testing, that a recitation immediately following study is not 
as adequate a test of learning as one held on the following day as would 
be the ease where the R-S sequence is employed. 

Among other things the numerous attempts to realize in practice the 
theoretical advantages of supervised study have brought to light the two 
following factors which have operated to seriously impair its efficiency : 

1) in the supervised study time following a recitation, pupils are not 
inclined to take the responsibility for study as seriously as necessary for 
effectiveness; they prefer to study for recitations looming ahead in the 
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next or subsequent periods or to postpone study until a time when the 
recitation is more imminent; (2) where the R-S sequence is used more 
often than not the recitation is permitted to run over into time theoretical. 
ly allotted to study. Largely because of these two developments the 
lengthened period has not proven in general to be much if any improve- 
ment over the short period where no regular supervised study is con- 
templated despite the increased cost of supervised study due to the con- 
sequent lightened teacher’s load. 

There have been those who have claimed that employment of the S-R 
sequence that the contribution to increased effectiveness resulting from 
placing supervised study just before recitation thereby utilizing the in- 
creased interests of pupils and by making certain that the time theoret- 
ically allotted to supervised study is actually realized, will more than 
offset the disadvantages accruing from employing the S-R sequence. 
It was the purpose of this group of experiments to furnish some objective 
experimental data pertinent to this question. 


THE EXPERIMENTAL GROUPS AND TECHNIQUE 

The experimental subjects were students in the University of Oregon 
High School, a six-year junior-senior high school. They pay no fees and 
are admitted in the order of application. The regular faculty is com- 
posed of somewhat superior, progressive, experienced teachers comparable 
to the professionally alive teachers in any city high school where a reason- 
ably adequate salary schedule obtains. 

Ten pairs of classes were employed as follows: 
History J2—(second semester, seventh grade, American history) 
Civies J4—(second semester, eighth grade, community civics) 
History J6—(second semester, ninth grade, medieval history) 
History S2—(second semester, tenth grade, modern European history) 
Civies S4—(second semester, eleventh grade, U. S. government) 
English J6—(second semester, ninth grade, literature) 
Mathematics J2—(second semester, seventh grade, unified mathematics) 
Mathematics J4—(second semester, eighth grade, unified mathematics) 


Mathematics J6—(second semester, ninth grade, unified mathematics) 
Science J4—(second semester, eighth grade, general science) 


The students were paired on the following points: Chronologica! 
age, Otis Intelligence Test scores (advanced examination), and an 
achievement test intended to furnish a measure of special aptitude. 

The subject matter tests employed were as follows: All history and 
civies classes, the Gregory American History Test, Part III, for seventh 
and eighth grades and high school; English J6, the Monroe Silent Read 
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ng Test; Mathematics J2, Stanford Achievement Arithmetic Examina- 
tions; Mathematics J4 and J6, the Douglass Standard Diagnostic Algebra 
Tests; Seience J4, the Ruch-Popenoe General Science Test. 

Owing to the great difficulty in arranging adequate pairs on three 
different eriteria, composite scores were calculated from the Otis scores 
and the achievement test scores in which the weightings were equalized 
by the method of equal variabilities, i. e., the scores on one of the two 
tests were multiplied by the ratio of the standard deviation of the scores 
made on the other test by the pupils of the class to be paired to the 
standard deviation of the scores made on the test the scores of which were 
being converted. 

The two experimental sections were so constructed from these pairings 

; to have one of each pair in each section and to result in sections of 
equal mean chronological age, composite score, and initial test score’ and 
of approximately equal variability in chronological age and composite 


score. 


CONTROLLING ALL BUT THE EXPERIMENTAL VARIABLE 

Each pair of sections recited at the same hour and in equally favor- 

le rooms. The teachers assigned to these sections were selected so as 
be of approximately equal prospective ability and were exchanged at 

e middle of the experimental period of ten weeks the influence of 
teacher efficiency being thus equalized between the two sections. Careful 
gs of the experiments, including absences and interruptions were kept. 
Beeause of the fact that each pair of sections recited at the same time, 
the only interfering circumstances affecting any pair of sections un- 
equally were absences and withdrawals. In these cases of withdrawals 
nd excessive absenees of more than five days, the absent pupil was 
lropped from experimental consideration and the pupil paired with him 
ther paired with some pupil in his section not originally paired or 


lr ypped also from consideration. 


MEASUREMENT OF RESULTS 


Objective tests requiring from 80 to 110 minutes were prepared to 


cover the work to be taught during the experimental periods. These 
initial tests were given the two days just preceding the beginning of the 
experiment. Similar, but not strictly comparable, objective tests were 
devised and given as final tests. These tests included such types of items 


*To be discussed later. 


) 
| 
| 
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as completion, true-false, multiple-choice, matching, analogy, and enu- 
meration exercises. In most instances the final tests included at least 
a few items taken from the initial tests. The initial and final tests for 
literature were identical. In the mathematics J6 sections, two forms of 
the Douglass Algebra tests were employed as initial and final tests. The 
reliabilities computed from scores of the half-tests of the students taking 
them by the Spearman-Brown prophecy formula ranged from .61+.09 to 
.94+.01 the middle 60 percent of them ranging from .75+.06 to .90 
+ 02. 

As a measure of the relative gain of the two sections was taken the 
following difference =( FT IT) of one section — 3(FT — IT) of 
the other. This of course is equal to the difference of the sums of the 
final test scores of the two sections since the sections were so constructed 
that the sums (or means) of the initial test scores of the two sections 
were equal. 

THE EXPERIMENTAL RESULTS 

The most important experimental results are summarized in the tabl 
given below. 

TABLE I 


CHANCES IN 100 THAT SUPERIORITY IS IN THE DIRECTION OF THE OBTAINED 
SUPERIORITY 


d SDa d Chances 
“SD in 10 

History J2 56 1.89 80 SRS RS 6 
History J4 70 1.97 36 SRS RS 64 
History J6 3.39 3.92 86 SRS RS 8! 
History S2 4.48 3.22 1.39 SRS RS 9 
History S4 3.92 1.95 2.01 SRSRS 
English J6 5.62 2.27 2.48 SRS RS 
Mathematics J2 1.50 86 1.83 RS< SR 97 
Mathematics J4 24 1.56 16 RS<SR 56 
Mathematics J6 1.14 96 1.19 RS<SR 88 
Science J4 4.38 5.11 36 RS<SR 81 
d=the difference in mean gain of the two sections, ie., (FT: FT:) +N 


SDa—the standard deviation of the difference in mean gain, as given by the following formula 
SDa= y SD*%u1+SD*uq—2ran we SD, SDoe 
d=the difference expressed in terms of its standard deviation 


SD 
In the last two columns of this table shown for each pair of sections 
was superior and what the chances are (by reference to some normal 
integral table) that the obtained difference is significant, i. e., that th 
apparent superiority of the section as indicated was not due to chane 
or sampling errors. 
Due to the fact that the initial and final test units were not compar 


able, that is, their relative values are not known, no comparisons can be 


made, such as will enable us to say that one sequence produced results so 
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many percent better than the other, except in two instances. In the 
literature sections, where the same test was used as initial and as final 
tests and in the sections in mathematics J6 section were two comparable 
forms of the Douglass Algebra tests were used, the following comparisons 
may be made. 

TABLE II 


PERCENTAGES OF SUPERIORITY OF THE MORE EFFECTIVE SEQUENCE FOR ENGLISH 
J6 AND MATHEMATICS J6 


GAIN IN 8-R GAIN IN RS SUPERIOR PERCENTAGE OF 

SECTION SECTION GAIN SUPERIORITY 
sh J6 699.6 626.6 70 11.17 
Mathematics J6 150.8 166.6 i6 9.59 


While no exact statement can be made as to the percentages of super- 
ority of the more effective sequence in classes other than those above 
view of the relative chances among the different classes that the ob- 
tained superiority of the apparently superior sequence is positive and 
n view of the fact that they are based upon groups of students of some- 


what similar numbers, it is not unreasonable to think of the percentages 
superiority as ranging from materially less than 10 percent (in the 
ses of mathematics J4, and history J4) to not much more than 10 per- 
t. The upper limit seems reasonable in view of the fact that the per- 
ntage of superiority in the ease of the English scores made on the same 
st was only 11.17 percent in spite of the fact that the superiority of 
the more effective sequence was established as not being due to chance 
we clearly than in the ease of any other class. 
It was interesting to note the relationship of the opinions of the 
teachers in charge of the experimental sections with reference to the 
lative effectiveness of the two sequences to the actual experimental 
ults. Table III below is pertinent to this point. 


eS 


TABLE III 
)MPARISON OF EXPERIMENTAL RESULTS WITH TEACHERS’ JUDGMENT BY CLASSES 


= TUDGME s 
EXPERIMENTAL TEACHER DGMENT 


RESULTS FAVORING FAVORING 
( e R-S = S-R R-S neither S-R 
tory x 1 1 
History J4 x 2 
History J6 x I 1 
History S2 x 2 
ristory S4 x 1 1 
English J6 x 1 1 
Mathematics J2 \ 1 
Mathematics .J4 x 2 
Mathematics J6 x 1 i 
Science J4 x 1 1 
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Table IV furnishes evidence as to the relative effect of the two 
sequences upon variability of progress within the section. This table 
should be interpreted similarly to Table I. 





TABLE IV 
VARIATION OF GAINS OF PUPILS By CLASS AND BY SEQUENCE 

Class SD gain SD gain SD air SD Chances that 

8-R R-S SD,-SD; difference 

SDar is real 

History J2 7.36 7.76 —.46 1.67 24 60 to 40 
History J4 5.70 5.09 61 1.35 46 68 to 32 
History J6 10.00 10.04 —.04 2.52 .02 51 to 49 
History S82 8.05 8.01 04 2.32 .02 51 to 49 
History S4 6.27 5.83 44 1.59 28 61 to 39 
Erglish J6 5.11 6.24 —1.13 1.57 .72 76 to 24 
Math. J2 3.72 8.84 —.12 73 16 56 to 44 
Math, J4 3.32 4.01 —1.29 1.17 1.10 86 to 14 
Math. J6 4.34 3.08 1.26 1.16 1.09 86 to 14 
Science J4 16.38 13.86 2.52 3.88 .68 75 to 25 


The relative effectiveness of the sequences for students of varying 
intelligence was investigated. Initial and final test scores of all pupils 
were expressed in terms of their own standard deviations. The mean 
relative superiority of the R-S Sections were each computed in these 
terms. As a measure of intelligence the mean Otis score of each pair was 
taken. Correlation coefficients between the superior effectiveness of the 
R-S sequence and intelligence are given in Table V. 


TABLE V 


COEFFICIENTS OF CORRELATION BETWEEN INTELLIGENCE AND SUPERIORITY OF THE 
RECITE-STUDY SEQUENCE 


r PE. 
Mathematics classes 115 .088 
Science and English classes 091 126 
Social Science classes 043 080 
All classes .063 57 


Examination of the ten correlation charts which are not reproduced 
here because of space limitations disclosed no significant relationship 
within any given range of intelligence. 


CONCLUSIONS 

1. It appears that neither the R-S sequence or the S-R sequence is 
superior for all kinds of classes. (See Table I) 

2. The S-R sequence is probably superior for classes in history and 
social studies and literature and the superiority becomes more marked 
as we progress from the seventh to the eleventh grade. (See Table I) 

3. The R-S sequence is probably more effective for classes in mathe- 
maties and general science, at least in the junior high schools. (See 
Table I) 
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4. Teachers’ judgments as to the superior effectiveness of two methods 

teaching should in all likelihood not be taken as conclusive. (See 
Table III) <A eareful study of the reliability of teachers’ judgments in 
such instances appears very desirable. 

5. Neither sequence seems to be connected in general with the pro- 
uction of larger variations of progress within the class than the other. 
See Table IV) 

6. Neither sequence seems peculiarly favorable for any given range 

of intelligence. (See Table V) 


While no scientific conclusive explanation can be given for the above 
results, a study of the logs of the instructors in charge of the experimental! 
sections and his own experience in observing and supervising supervised 


dy technique and practice leads him to offer the following hypoth- 


eses. In those classes where the effectiveness of study does not depend 
so directly upon specifie assignments better results will follow the use of 
S-R sequence due to two factors (1) the increased interest and con- 
quent higher level of attention in the study period immediately pre 
ling a recitation, (2) where the S-R sequence is used the student has 
beginning of preparation, has located his difficulties and is in a 

ptive if not a ‘‘seeking’’ attitude toward assistance. 
In mathematies classes, study should probably follow presentation 
ew material, rules, and procedures as soon as possible, while the ex- 
tions and directions are fresh in the minds of the learners. In 
ldition the R-S sequence provides a short tryout for the assignment 
e] 


+ 


often results in additional desirable explanations to the class as 
ole or to individuals needing it. 
Not to be ignored is the quite likely explanation that in subjects in- 
ving memory largely, reeall and repetition within the next hour, as 
the case of the S-R sequence, will serve to fix before the setting in of 
rapid decline in the forgetting curve gets under way. In other words, 
tudy followed immediately by recall and recitation as in the S-R 
quenee is probably a better arrangement for acquiring history, civics, 
literature facts, concepts and information, than study followed by 
ll and recitation 2314 hours later after the inroads of forgetting 
vn by many previous experiments to be large have operated to wipe 


ut much of the results of study. 
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EDITORIALS % 
SOME MATTERS OF EDITORIAL POLICY 


Is the Journal of Educational Research too technical? Is it techni- 
cal enough? Should the journal be for research specialists or should it 


tea tatateraipa 


be for teachers, principals, supervisors and superintendents interested 
in seientifie education? Should the Journal of Educational Research 
aim definitely to modify school practice? 

Dr. B. R. Buckingham, in the first issue of the Journal of Educational 
Research (January, 1920) made the following announcement of policy: 


‘*The Journal of Educational Research will be devoted to the uses 
which have been made and are capable of being made of the results and 
methods of investigation. It will emphasize applications rather than 
abstractions, and practice rather than theory. It will not avoid the theo- 
retical altogether, and it will present some technical articles. But such 
articles will be most weleome when they have obvious practical implica- 
tions. Research for the sake of research we shall leave for others. What 
uses the teacher and supervisor have made or may make of the findings 
of the experimentalist, what methods are transferable from the labora- 
tory to the classroom and office, what workable means of meeting com- 
mon needs have been developed under actual school conditions—these 
will be our chief concern. We shall seek contributions not only from the 
leaders in research but also from their followers, the practical school 
people. 


are quite generally abstract, and ‘tentative conclusions’ are often merely 
speculative. Some of us wonder whether all this is necessary—whether 
a writer may not be scientific without being impractical or profound 
without being obseure—whether the time has not come when some of these 
results may be restated in usable form—whether we do not now possess 
the materials for studying some problems long enough and hard enoug! 
to reach conclusions which are fundamental and satisfactory. 

‘‘The attempt will, therefore, be to permit practice to catch up with 
theory. The Journal will endeavor to exemplify sound theory and it is 
hoped that its technie will be adequate. But we believe its theory and 
technie will be best exemplified when they are employed in the report- 
ing of practical investigations and in the formation of practical results.’’ 


Six years later, Dr. E. J. Ashbaugh returns to the problem in an 
editorial, ‘‘ Are We Too Highbrow?’’ 


246 


‘*Theory has outrun practice in our educational writings. Results | 
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“*a)) 


‘‘For whom should educational magazines be published? What is 
their mission? Is their primary object simply to report or is it to re- 
port in order to modify practice? Are they publicity agencies for their 
contributors or are they service instruments for the benefit of the school 
hildren of the country? How much technical education have we a right 

assume on the part of readers or to what extent may these magazines 
be expected to be the source of this technical education? 

‘‘As we examine educational magazines we find evidence of various 
jlicies. Some are so free from technical terms that it is safe to assume 
t anyone can read them—even university presidents, and that with- 
t a dictionary. Mast of these relate personal experiences and opinions 
d savor little of contact with research and the scientific movement in 


US 


ee 





lie 


he education. They are apparently written for teachers who are neither 

ial | trained nor eriti¢al. Others show evidence of an awareness of present 
7 . . pate . . . . . 

vy: eed cational tendencies. Their articles report the results of a variation 


wedure which was to some extent at least checked by control meth- 
Mods. The language is relatively simple though the statements at times 
dogmatic. Some discrimination is evidently expected of the reader 
all the data are practically never presented. A third type of 
f= magazine is also available. This type fairly bristles with technical 


NCR RIS wr ee 
_ 


. rms—quartiles, percentiles and ogives, P.E.’s and sigmas, correlations 
multiple and partial, and regressions. Statistics by which to validate 

; cedure and test the results have become an important instrument in 
BS tifie educational work, and, therefore, the authors, having made use 
: f statistical procedure, feel the necessity of reporting all of it in the 


rts of their experiments. 

Whenever any one develops a new formula or a new use for an old 
mula, technical workers will certainly be pleased to know about it. 
Every worker should safeguard himself by every means at his disposal 

the conduct of an educational experiment and the interpretation of 
s findings. On the other hand, when reporting investigations which 
re of value to public-school people, it is certainly a mistake to burden 

report with a technical vocabulary which few understand. Research 


’ / 
ee ee 


nd lings are valuable only as they modify educational practice.’’ 
g This question of editorial policy is bs no means a new one. The edi- 
; il board is anxious to have the Journal of Educational Research serve 
ie the cause of scientific education as effectively as it can. To this end it 
s thought that it might be helpful to get an expression of opinion from 


ts readers. You have already received or will receive in the near future 
questionnaire asking certain questions about matters of policy. We 


} should be glad to have you assist us by filling out this questionnaire and 


returning it. We shall present a digest of this material in a later issue 


f the Journal. The questions asked are: 
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QUESTIONS 


(Check in these 
Columns) 








1. Should the JOURNAL OF EDUCATIONAL RESEARCH 
aim definitely to modify school practice? (To do so, would 
probably mean that the JOURNAL should attempt to serve 
teachers, principals, supervisors, and superintendents, who are 
interested in the application of scientific methods to school 
problems.) 


2. Do you think that a journal of research might be more scien- 
tific and less technical? (It has been suggested that the readers 
want not less science, but more care exercised in making the 
descriptive and interpretative portions understandable to teachers, 
poe supervisors, superintendents, etc., those not trained 
in technical research.) 


3. As a contributor to the JOURNAL OF EDUCATIONAL 
RESEARCH, do you prefer to write: (1) for the research spec- 
ialist alone, or (2) for teachers, principals, supervisors, and sup- 
erintendents interested in scientific education, or (3) compre- 
hensively enough to interest both groups? 
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4. Should reports of research be published which seem to follow 
a reasonably sound experimental technique but which are based 
upon a relatively small number of cases, or seem preliminary in 
character or not sufficiently conclusive to serve as a basis upon 
which to modify school practice? (A superintendent of schools 
recently remarked that he did not object to the reading of tech- 
nical articles—as a matter of fact he preferred them—but he did 
object to spending an hour or more working through such an 
article to find in the end that it led nowhere, that is, led nowhere 
from his point of view. 


| 


| 
| 


a | No ff 











give special attention during the coming year? (Such as (1 
curriculum research, (2) experimental studies of teaching and 
learning, (3) measurements applied to character education, (4) 
guidance, (5) child study, (6) special education, (7) supervision, 
(8) administration, etc. 


Yes | No 
(Write Nos. here 
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{ 
aK 
AT coonsesaosennenantienoentt S) | 
‘i : 
_ S ‘Ds, PercivaL M {bility Stand mum expectation for gifted students but : 
i Standardized Achievement rather a series of scores that do not im 
ie | - . 
4 ] ; in the High School, New York, pose too rigid standards upon the less 
hers College, Columbia Univer gifted nor too low vet attainable stand } 
y 1927. x 91 pp. ards for the gifted students. 
: I 
ia . In the monograph are included tables 
: ntil the publication of the present P a I , ; - is 
; a giving mental ability standards for 
graph, both the value and the . 
‘ December Ist, March Ist, and June Ist 
eance of the use of standardized . Be ee . 
for the American Council French Test, 
ts in the secondary school have been H R li 1 . ll. 
aed : : : aggerty ading Test Sigma , Hotz 
isly limited. The variability of d ae b 6 
‘eee Equations and Problems Tests, Powers 
rh-school classes in mental ability has ; é ; 
; 4 Chemistry Test, Ruch-Cossman Biology 
d the comparison of achievement ies : Nees 
lest, Ruch-Popenoe General Science 
lass scores with one another or with bi a 
a . : : Test, Schorling-Sanford Geometry Test, 
rds of little significance. In : . 
; - , Van Wagenen American History Seale 
al differences in mental ability wh ' ' 
S3, White Latin Tests. 
n high-school classes has likewise re 
. or interpreting the test svores of 
apped any attempt to interpret ' hod 
. individua uplis two methods are pro 
scores made by individual students. i E $ I 
posed. One simply shows the difference 
+} ea ant fror Teachs c% —_ 
, :' | Teachers 2 between the mental ability norm for the 
- { nad ie CoupnpeygTt i) » Tr 
i und th Operation of the prin student and the student’s score in terms 
and acher ' arly , : a 
a teachers in nearly forty of the test score. The second method 
ary sch _ Dr ymonds under expresses this difference in terms of the 
the task of determining mental mental ability unit, thus making differ 
ty norms for a group of standard ences from one test to another compar 
i tests and seales for secondary able. 
schools. These mental ability norms were While the plan of Dr. Symonds as 
ned by having both the Terman sumes a constancy of units of measure 
ip Test of Mental Ability, and the ment which is not yet attainable with 
ement tests given to several hun the usual standardized test, it provides a 
ired high-school students a constant reasonable and very valuable working 
which had been worked out pre- basis for making a more significant use 
isly by Thorndike. The method of and a more adequate interpretation of 
i ul-pereentiles finally decided upon for achievement test scores in high-school 
| | determining the standards gives neither subjects. 
| average scores for pupils of varying M. J. VAN WAGENEN 
ees of mental ability nor the maxi- University of Minnesota 
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LOWMAN, C. L., CoLesock, CLAIRE, and 


Cooper, HazeL. Corrective Physical 
New York, A. 
S. Barnes and Company, 1928. xiv +4 
521 pp. 


No current trend in physical education 


Education for Groups. 


is of greater significance than that of 


meeting the health needs of individual 
children but no development of the 


public school program could catch phys- 


ical educators more unprepared to meet 


their opportunity Therefore, the book 
by Dr. Lowman and his associates 
hould find an eager reading publie, for 


it provides not only descriptions of cor 


reetive exercises for kyphosis, lordosis, 


foot 


activity 


scoliosis, weaknesses, ete., but also 


definite schedules adapted to 


groups of different degrees of malad 
justment, and detailed accounts of gen 
eral and specific methods of teaching. 
As this volume was prepared by an 
orthopedic surgeon, with the aid of ex 


perienced public school physical educa 
corrective 
of 


who 


tors specializing in gym 


contains much 
the field 


specific instructions 


nastics, it 
to 


practical 
worker desires 
detailed 


conducting 


value 
and advice 
and ele 


of Un 


fortunately, however, the subject matter 


in organizing an 


mentary type corrective work. 
is so clumsily put together that all too 
diffieult to 
corrective activity material from useless 


Much other 


included in this volume, such as elemen- 


often it is separate useful 


general outlines. material 
tary physiology and kinesiology, and in- 
formational hygiene programs for phys- 
should 


related more closely to activities or be 


ical edueation classes either be 
treated more comprehensively in another 
book. 
Moreover, the chapters on organiza- 
tion betray a lack of clear planning or 


of the field of 


measurements. For example, 


educational 


the 


knowledge 


JOURNAL OF EDUCATIONAL RESEARCH 


‘*Reilly system of classification’’ is ad 
vocated as bringing together those of 


equal abilities and physiological ma 
turity. There are now several other 
ways of determining physical and 


almost double 
More 


over, this system, instead of bringing to- 


athletic ‘‘abilities’’ with 


the validity of the Reilly system. 


gether those of the same physiological 
characteristics, tends to bring together 


(in the senior high school especially 
most heterogeneous groups. 


There is a cloud of chaff, a lack 
organization of material, and some err 


in this volume; but the notes on equiy 


ment, the deseriptions of exercises, and 


the corrective schedules, coupled with th 


great need for treatises on correctiv 


physical education, probably justifies its 
publication. 
FREDERICK RAND ROGERS 


York State Education Departn 


New 


CRAWFORD, CLAUDE C. 
Study. New York, 


Company, 1928. 


The Technique o/ 
Houghton Mifflir 
353 pp. 

In The Technique of Study, Mr. Craw 
ford has provided a splendid service for 
college students. The mass productior 


methods of college  instruction,—the 


stricter requirements for staying in col 
lege after one has passed the gate, th 
grading systems, 


mathematical ways of 


the mind-reading devices of tests and 
new-type ‘examinations,’’ all such 
make demands upon students for a 


quality of workmanship and_ technical 


ability to handle subject matter un 
dreamed of in an older, more human 
time. If a student gets into college by 


virtue of a high I. Q., a good high-school 


record, and generous character recom 
mendations, he may stay there only bj 
virtue of his ability to adjust himself 
to the college courses in a no worse than 


**C’’ equilibrium. If the balance get 
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vy toward the ‘‘D’’ end of the scale portunities for collecting material and 

tomatically down and out. trying it out which were at Mr. Craw 

g how to study is usually the ford’s disposal, he might have given some 

id most urgent need of the fresh help in special study methods For ex 
The sophomore and even the ample, the material in the chapter on 

yy may in their increasing wisdom ‘* Memorizing’’ makes practically no ad 
the value of better and speed vance over James’ chapter written thirty : 

1€s. eight vears ago. Verbatim memorizing 

Crawford does that which has f dates may have gone out of fashion 

Whipple’s little manual a_ best but students who desire an A grade want 
ve e years. He arranges his t know what to remember and how to : 

the form of definite practical recall it at the time of need. Have his 

He says in the preface, ‘‘A torv teachers, or mathematics teachers 

| i be made up largely or language teachers contributed noth 

‘ . the g to the peculiar techniques of memo 

hou Material of the wh rizing in their special fields? Special 

ild be treated as next in im methods were omitted no doubt with ds 

though it should be definitely liberate intention, but the how and the 

former type of material rf? why, as well as the monotony of pres 

is for these rules is thoroughl entation would have been much illumi 

AT n¢ as M Crawfoay nated vy the inelusion of nerete illus 

site ps rhe Y l I . . t tT! he rich sour s of the 

} } 3 } } sul el eI l 

} terial to eareful experi The soci aspects of study are not 

t [t has bee ied out given s t space to help the student 

{ l t ag 1 many solve the extremely difficult problems 

tuations. Nearly all the general presented to him daily In most of the 


edures considered, ‘* Tak chapters some one of the paragraphs ré 


tes,’”’ ** Listenin to Lectures, fers to group study, but the student is 


lexthbooks,’’ ‘‘ Memorizing,’’ not shown how to win control over grouy 
aking Tests’’ are a few of tl situations either for supplementing his 
r headings Each chapter con private enterprise or for directing o 
s a large number of rules carefull coo} iting with others. Learning t 
i, a set of quest : nd ex ** think together’’ should become one 
veral type tests « the content, creat objective of mass education in a 

bibl rraphy lemocracy, an objeetive consciously gl 
S book lends itself admirably for ing both teachers and students. The dis 
cher as a textbook in met cussion method of study in college pro 
f ly. For students working alone vides the habit tendencies which must 
t a supervisor or class the bool work over later into our civie life and 

ful and suggestive machineries of governmen 

en, it seems necessary to mention Finally, Mr. Crawford scarcely tapped 
tain inadequacies, it must not be in the reservoir of practical devices for 


1 that what is missing diminishes ‘*eoncentration’’ (a student term), for 


value of what is given. However, memorizing, for test-taking, indeed, for 


ean but wish that with all the op all the everyday study situations pre 








body. ** How do 


. 


' stuck 
you do it?’’ is a fair question to ask of 
every student who maintains a high grade 
level. The reveal of 


practical experience applied to the spe- 


answers a wealth 


cific school situation. To be sure, much 


of this material has not attained the dig- 


nity of ‘‘scientifie’’ psychology, it is 
me rely practical, it works. 
The student is not given a glimpse into 


the pros and eons of muscle tensions, of 


suggestion, of fatigue and recovery, of 
emotion, of indirect mental vision, of 
atmosphere, although within these ex- 


periences one may find ‘‘ those reserves of 
power and capacity and those latent pos 
sibilities of 


thinking that are 


education and constructive 


present in all of us’’ 
as Dr. Herrick so 


‘How to Stu ly , 


vividly points out. 
must get deeper and far- 
ther than the orthodox, the traditional 
psychology of James and Whipple and 
Edwards Thom if it is to of 
vital the in his tre 


mendous problem of 


and be 


as 


service to student 
succeeding in college. 
JESSIE ALLEN CHARTERS 


Ohio State University 


of 


for School Purposes 


Cavins, L V. Standardization 
i{merican Poetry 
Chieago, The 


1928. x 


University of Chicago 


134 pp. 


Press, 


Dr. Cavins presents an interesting and 


worthwhile addition to the studies al- 
ready made in classifying literature for 
school use. His declared purpose is to 
arrange poems for classroom use on the 
basis of their appeal to certain inter- 
ests of pupils according to content. The 


study furthermore presents scientific evi- 
the thought lan- 
that children have 


dence concerning and 


guage difficulties 

actually found. 
After furnishing a digest of the pre- 

that have been made in 


vious studies 


the placement of poems in the school 
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curriculum, the findings of a new in 
vestigation of the placement of sixty 
four standard selections in the courses 


of study now in use in every state of 
the Union is presented. 

The book contains a carefully ana 
lyzed symposium (by ten English schol 
and textbook of nationa! 


importance) on the ‘‘ factors to be con 


ars writers 


sidered in placing a in our ecu 


It explains the construction 


poem 
rieulum.’’ 
of 
perience of these experts. 

The 
in determining the validity of the tests 
in arranging poetry for school uses. He 
furnished evidence to show that the 


tests as based on the advice and ex 


author has experimented widely 


has 
principles upon Which the tests are based 
are psychologically sound and in accord 
regard to what 


their stud) 


with aecepted aims in 


children should derive from 


of literature. R. FE. Gree 


Ohio State University 


Renaissance Student Life: The Paed 
logia of Petrus Mosellanus. Trans 
lated from the Latin by Rosert 
FRANCIS SEYBOLT University of Ill 


nois Press. 


Professor S« vbolt has done a co! 


siderable service in translating this boob 


and making it accessible to students of 
the history of education. It 
book which played a significant part 

the The 


Petrus was 


is a text 


in Renaissance. 

Paedologia of Mosellanus 
first published in 1518. It 
keenly felt human need for a lesson book 
from which one might learn to convers 
in the style of Cicer 
Mosellanus, distin 
guished humanist who belonged to the 
of Reuchlin, Melancthon and 
Erasmus. He served with distinction % 
Professor of Greek at Leipzig and was 
of that 


ushering 


answered 3 


in Latin, and 


its author, was a 


company 


rector university. 
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The conversations of this book are 
aluable in two respects. They tell us 
w Latin was studied in that educa 


nally energetic morning of the mod 
world, and since they are conversa 


s between teachers and scholars and 


etween scholars and scholars about 
ry day interests and school affairs, 
I are wonderfully informing as to 


educational practices of that day 


Next to the story itself nothing is so 
ch needed for understanding the 
history of education as an abundance 


f detailed original material to make 
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that story concrete and wholly under 
standable. That is the great value of 
this little book. Professor Seybolt has 
spent upon it no little skilled labor and 
is to be congratulated, first, upon his 


willingness to put himself in this fash 
fellow students 
the 


ion at the service of his 


of education; and second, 
worth of the 


he has made. 


upon 
illuminating little volume 
Here the 
stairways up which our ancestors climb 
ir effort to master the Latin of 


which is one of 


ed in the 
Cicero. MoorE 


inacles 


ERNEST C 
U naive situ of California at Los 
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Wy) Address all communications relative to research abstracts and 
bibliographies to Editor A. S. Barr, University of Wisconsin, 
Madison, Wisconsin. 


**Edu- 


(Erstes 


First bibliographical booklet. 
eational Literature’’ 1924-26. 
bibliographisches Heft) (Die erziehung- 
swissenschaftliche 1924-26). 

Second bibliographical booklet. ‘‘ Edu 
(Zweites 


Forschung 


eational Literature’’ 1925-26 
(Die erziehung 


1925-26.) 


bibliographisches Heft) 
wissenschaftliche Forschung 
Third bibliographical booklet. ‘‘ Edu- 
‘Total Bibliog- 


(Drittes bibliog 


eational Literature’’ 


’* 


raphy of Pedagogic. 


raphisches Heft) (Die erziehungswiss- 
enschaftliche Forschung. Paedagogi- 
shehe Gesamtbibliographi« 

Academy of Sciences of General 
Utility at Erfurt. 


tion 


Section for Educa 
(Akademie gem 
einnuetziger Wissenschaften zu Erfurt.) 
fuer Erziehungswissenschaft 
nd Jugendkunde.) Edited by Dr. 
ArnTHUR HoFrrMAN, Publisher Kurt Sten- 
ger, Erfurt. (Verlag 
Erfurt.) 
The 


and Adolescence. 


(Abteilung 


Kurt Stenger, 


f American Research 
should be called to the 
They offer 


| 
articies, 


attention « 


workers, above 


publications. a comprehen 


sive treatises, 


bibliography of 


dissertations, and books written in the 


German language dealing with educa 


Academy at Er 


tion. Experts of the 

furt and of the ‘‘ National Committee 
on Education and Instruction’’ (Deut 
scher Ausschuss fuer Erziehung und 


Unterricht) at Berlin under leadership 


of Dr. Hoffman are responsible for these 
carefully 


means of 


publicatior 8 By 


‘A 


worked out schemes of classification, 
keys, and other bibliographical devices, 
literature is made available 
of the third booklet is as 


is similar to the other 


educational 
The 


follows: 


content 
(and it 
two): (1) German scientifie educationa! 


books, 1927. (2) 


cational periodical literature. 


Bibliography of edu 
(3) Ger 


man university publications, bibliog 
raphy of doctor dissertations, ete. (4 
Educational books in the English lar 


guage, 1926. (5) Research and the 
teaching of education at universities and 
other institutions of higher learning in 
the German language territory, winter 
semester 1927. (6) The eurricula at the 
pedagogical academies of Prussia, winter 
semester, 1927. 


HENRY LESTER and WRIGHT, 
WENDELL WILLIAM. ‘‘ Second Revision 
of the Bibliography of Educational 
Measurements, ’’ Bulletin of the 
Bloomington. 
1928. 


SMITH, 


School of Education, 


Indiana, Indiana University, 
This bulletin is the second revision of 
Educationa! 


Burea' 


the Bibliography of 
Measurements compiled by the 
of Co-operative Research, Indiana Uni 
published in 1923, revised in 
The bulletin contains 
well-ar 


versity, 
1925 and in 1928. 
and 


a complete, up-to-date, 


ranged list of tests. 
WAGNER, Dr. Junius. ‘*‘The Scientifi 


of Present Day Pedagogy.’’ 
Wissenschaftsmethodisehen Rich 


Trend 
(Die 
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RESEARCH 


ven der gegen waertigen paeda 
gik.) Academy of Sctences of Gen- 


Section for 
( Aecad- 
Wissenschaften 


al Utility at 


and 


Erfurt, 
Education Adolescence. 
emie gemeinnuetziger 
Erfurt, abteilung fuer Erziehung- 
1928. 


swissenschaft und Jugenkunde) 


Wagner, the differ 


methods are 


According to Dr. 

educational research 
ved from three mental attitudes: the 
rieal, the speculative, and the criti 
topics are discussed 
The 
ction of empirical pedagogy. 2. The 


3. The 


‘he following 


_ 


empirical 


pedagogy: 1. 


eal method in pedagogy. 


mental procedure. (A) The di 
ul and the psychological em 
s (a Foundation by Lay and 
[ uu b) Pedagogical psychology 
ts problems. B) siologiecal em 


C) Sociological empiricism. 


ler eritical pedagogy we find thr 


‘SB. 


@ titles: 1. The nature of criti 

2 Methodological problem. 
enological and the descriptive 
hod 4. Normative pedagogy. (a) 
g teleology. b) The value 
in pedagogy. (e Task and 
g anee of the pedagogical theory of 
(d The pedagogical norm 
es Neither empirical, speculative 
critical methods of attacking edu- 
nal problems can lead to a complete 


system of education according to 


EXPERIMENTAL STUDIES REI 


OF OTHER 
LEARNING 
Role of 


PSYCHOLOGY OF 


vy, M. C. **The Articula 





n Memorizing,’’ Journal of Ex 
tmental Psychology, XI (August, 
28), 306-312. 
\ stud of the advant if any 
rticulation in memorization. The 
t-matte consisted of two 20-wore 





Dr. Wagner. 


WEIDEL, 


training was made by 
lishing 
Bonn, Kiel, Frankfurt a. M., and Elbing 


teache I 


teacher 


? 
as acaqaemy 


pendix contains 


examination 


veved no 


in school 
in addition to 51 


level were used. 
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They supplement eacn 


other and must in turn be supplemented 


by the philosophical method. 


New 
(Die 
No. 


{cademy 


‘* The 


Prussia’’ 


KARL, PROF. 


Teacher Training in 
neue Lehrerbildung in Preussen) 
13 of the publication of the 
of sciences of general utility at Erfurt 


(Akademie 


chaften zu 


issens 


Edu 


gemeinnuetziger V 
Erfurt) 


Section for 


cation and Adolescence (Abteilung 
fuer Erziehungswissenschaft and 
Jugendkunde). 1928, Publisher: Ver 
lag Kurt Stenger, Erfurt. 


The first attempt to reorganize teacher 


Prussia in estab 


new pedagogical academies at 


Teachers have first to graduate from a 
gymnasia, then visit for two years a 
edagogical academy and pass the first 


examination. After teaching fo 


four years they have to pass the second 


examination. Hereafter thev 


“an be appointed as state officer. 


The course of study, aims and ol 


methods and procedures, ete.. 


are fully discussed by Professor Weide}, 


director at Elbing. An ap 


the actual course oi 


study for the last year, the new regula 
tion for the first and second teache 


in Prussia, the equipment of 


the academy, ete. 


-ORTED IN CURRENT ISSUES 
MAGAZINES 


lists of three-letter syllables 


that 
Three 


con 


obvious meaning. hun 


dred and twenty-seven subjects ranging 


from the second to the ninth, 


students of college 


The results are briefly 


summarized by the author as follows: 


1) The average result from 


every one 
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of our groups shows that the syllables 


were more economically learned with 


full than restricted articulation. (2) 


The girls make better seores than the 
boys in both kinds of learning. (3) 


The pronounced individual differences in 
relative number of syllables learned 
hy the 


experimental 


the 


two methods point to a need of 


determination for each 


pupil of the better method for his use in 
memorization 1) The errors made by 


the first-deseribed method are slightly in 


excess of those made by the latter. They 
are about equally numerous in the lower 
and in the upper grades, and more 


numerous in the intermediate grades. 


‘*Analysis of Dif- 
Elemen- 
(Sept. 


Leo J. 
Decimals, ’’ 


BRUECKNER, 
ficulties in 
tary School 
1928), 32-41 


The 
X XIX 


Journal, 


‘his article contains the results of a 


study of the errors made by pupils in 
Grades VI, VIT, and VIII in analytical 
four decimal 

The basis of the analysis was a study of 


the 


pupils in these grades. 


teats on the processes in 


written work of some three hundred 
The writer was 
concerned with the types of 
difficult the 
Tests were given (these 
and the 


primarily 


exampk found most and 


fnuses of error 


are reprodueed in the article) 


incorrect examples examined to deter 


mine causes of error. The total number 


of different kinds of 


errors isolated was 


114; the kinds of errors were more 
numerous in multiplication and division 
than in addition and subtraction. They 
were distributed as follows: general 


difficulties in reading, writing, and con 
version, 33; 15; 


difficulties in subtraction, 14; difficulties 


difficulties in addition, 
in multiplication, 26; and difficulties in 
The tables reproduced in 
the 
The author draws the 


division, 26. 
classification of 


fol 


the article contain 
8,785 errors. 


onelusions: 


owing ¢ 
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1. Many pupils do not have adequate 


concepts of the numerical values of 


decimals. Only two other types of errors 


occurred as frequently, namely, mis 


decimal point in division and 
the 


placing the 


misplacing decimal point in multi 
plivation. 

2. Many 
spelling of 
form; for 


** hundredths. ’’ 


errors were due to the mis 


the decimal written in word 


example, ‘‘hundreds’’ fi 


5. Failure to place the decima 
point correctly was the greatest cause of 
errors in addition, .3 5+ 8 16 be 
ing the most common type. 

t. The number of errors in addition 
due to inaccuracy was about half as 
great as the number of errors due to 


the misplacing of the decimal point. 
5. The 


traction were in borrowing and in the 


greatest difficulties in sul 


placement of the decimal number in the 


subtrahend. ' There were few errors due 
to imaccuracy. 
6. The difficulty in ft 


multiplication of decimals was the 


major 


placing of the decimal point or its cor 
plete omission. 
7. There 
inaccuracy in multiplication. 
8. The of 


division misplacing of th 


were many errors due 


major causes errors 


the 
decimal point, faulty placement of zeros, 


were 


omission of the decimal point, and in 


necuracy. 


FENNELL, ETHEL L. ‘‘ Reeurring Words 
and their Relations to Difficulties 
Comprehension, ’’ The Elementar 
School Journal, XXIX (Sept. 1928 
$2-53. 


The study of reeurring words of dif 
Th 


article points out that a probable reason 


ferent meanings, found in readers. 


in the comprehension 
the fact that textbook 
effort to familiariz 


for difficulty 
reading lies in 


writers in their 
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the best 


children with certain words have pro 
ed for numerous recurrences of these 
rds but have disregarded the fact that 
frequently the words recur with entirely 


rent meanings. 


FRANZ. ‘‘ Testing the Per 
School 


\RSCHAU, 


’ 


Knowledge. ’ 


nanency of 
Pruefung des Dauererfolges der 
Schularbeit.) Zeitschrift fuer paeda- 
gogische Psychologie, experimentelle 
Paedagogik und jugendkundliche For 
hung. (July-August, 1928), 351-356. 


Investigations of the knowledge and 
skill of young school graduates in in 
justry, a number of investigations are 


ferred to, two of which are sum 
arized below: 


One was made at the University of 


Jena by H. Meister on ‘‘ The Retention 


School Knowledge in Its Relation to 
e Personality.’’ (Die Retention des 
Schulwissens in ihrer Beziehung zur Pe 
hkeit). and another at the Uni 
sity of Mueller 


ig on **School Knowledge and Skill 


Koenigsberg by 
Graduates. (Schulmaessiges Wis 
nd Koennen bei Schulentlassenen. ) 
Mr. Meister examined 20 male persons, 
25 years of age The time since 
g school was from 6 to 11 years 

He put 162 oral questions on all ele 
tary subjects to them, and evaluated 
he answers from 0 to 3 points using 
possible criteria 
He found that they 


etic 57 pereent, in German 61 percent, 


answer as 


retained in Arith 


pereent, in Geography 6? 


‘ 


n History 5 


percent, in Natural Science 52 percent, 


nd in Religion 32 percent of the sub 
taught Thus 
uined about 50 percent of the facts 


et matter there re 
ght in elementary grades. 
Mueller-Jung examined 100 girls at 


jing vocational schools who had left 


lementary school 1 to 3 years hefore 
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He secured the following results: 
Arithmetic 54 percent, History 35 per 
cent, 


Geography 47.5 percent § and 


Natural Sciences 56 percent. The aver 
age on the later mentioned subjects was 
48.1 percent. Both investigations con 
clude that graduates of the elementary 
school retained about 50 percent of the 
facts that had been taught them. These 
findings open the elementary school to 
criticism in regard to the efficiency with 


which facts are taught. 


TINKER, MILES A, 


Duration, Pause Duration, and Read 


‘*Eye Movement 


ing Time,’’ Psychological Review, 


XXXV (September, 1928), 385-397. 
The purpose of this study was: (1) 
To furnish new data on the extent and 
duration of eye movements in reading; 
2) To show the variation in the ratio 
between eye movement duration and 
total reading time in various kinds of 
material. A photographic technique 
was employed to record eye movements 
Four kinds of material were used: (a) 
Thirteen lines of easy scientific prose; 
elementary 


(b) four paragraphs of 


Chemistry; (c) Two paragraphs of 
Algebra; and (d) Four lines of Alge 
The authors coneluded: 


yielded 


right 


braiec formulae. 
l. That his 


movements to the 


study reading 


which varied 


»° 


from 1° (formulae) to 2 (prose) ; 


movements to the left being from 13° 
(prose) to 16° (formulae). 

2. For all materials, less than 10 per 
cent of the reading time was taken by 
eye movements. The total average time 
taken for eye movements was 5.9 per 
cent of the reading time. This allows 
about 94 percent for perception, 1. e., 
pauses. 

3. The 


duration and reading time bear a con 


ratio between eve movement 


stant 


reading at 


relationship to the 


or 
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titude. It was found that the more care 
ful and analytical the reading the smai- 
ler this ratio, when the mental reading 
involved in the 
the 


processes reading wer 


more complex reading pause con 


sumed a 


the total time. 


relatively greater percent of 


TINKER, MILEs A. PATERSON, 
DonaLp G. ‘‘Influence of Type Form 
on Speed of Reading,’’ The Journal of 


XII, 


AND 


Psycholog uv, 


359-368. 


Applied (August, 


1928), 
Forms A and B of the Chapman-Cook 
Speed of set in 10 


Reading Test were 


point old style in each of the three 
forms, namely, all capitals, lower case, 
and italics. Eight groups of 80 each 
were tested, or 640 in all. The results 
are summarized by the authors as fol 


iows: 
1. A speed of reading record was ob 


tained for texts set up in roman lower 


can Com- 


, italic, and all capital letters. 
parisons were made between all capital 
lower case text. There were 640 


versus 


subjects used in the experiment. 

2. The text in lower case letters was 

ad 13.4 pereent faster than that in 
all eapitals. The difference in favor of 
the lower ease text is highly significant. 
There were 4.74 words read per second 
of the all capitals and 5.38 words per 
second of the lower case text. 

3. The text in lower case letters was 
read 2.8 percent faster than the italics. 


r 
The direction of the difference was con 
sistent for all sub-groups. The subjects 
id 5.08 words per second of the italics 


} = oo 


und 5.22 words per second of the lower 
se text. 
1. The following conclusions seem 


ustified by these data: (a) Text printed 
in all eapitals is less legible than ma- 


terial in lower letters and slows 


cease 


lown 


speed of reading to a very marked 
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degree. (b) Text printed in italica is 


slightly less legible than material in 
lower case letters and decreases speed 


of reading somewhat. 


MISCELLANEOUS 
Dr. H. 
Means of 
Aufsatz im Dienste der Schueler 
Zeitschrift 
sche Psychologie, experimentelle Pae 
und jugendkundliche For 
XXIX (April 1928), 199-205 


a previous study of the gran 


BECKMANN, ‘*The Composition 


as a Selecting Pupils’’ 
Der 
auslese ). fuer paedagog 
dagogik 
schung. 
From 
matieal development of the written ex 


pression of pupils of grade-school ag 
the author developed a scale which lh 
this This 


which is a seale showing the frequency 


uses in investigation. seal: 
with which different parts of speech ar 
used in different grades is given below 
(reported on the above Journal No. 6 


1927, p. 274) 
QO ae > = a a 
» a & > s 2 
> = = < z 2 
: > 3 8 3 
s ° . - c 3 

Zz . S S 

5 4 

e 

i] 
l 0.0% 1.1% 17.7% 4.9% 0.32% 
2 1.1% 2.5% 15.7% 5.0% 0.6% 
3 1.2% 2.7% 12.2% 6.8% 0 9% 
4 2.1% 3.6% 8.6% 7.0% 1.5% 
5 2.7% 3.7% 9.4% 7.6% 1.6% 
6 5.7% 4.2% 7.3% 8.5% 2.2% 
7 5.5% 4.89, 5.99 10.0% 3.3% 
8 5.5% 5.8% 4.4% 10.6% 3.9% 


This table indicates that 0.0 percent 
of the written language of the pupil of 
the first grade consists of abstract 
nouns; 1.1 percent consists of adjectives 
17.7 percent consists of adverbs, etc. | 
the herein 2:: 


fourth-grade 


investigation reported 
pupils wrote a eompositio) 
which Mr. Beckmann evaluated by first 
determining the frequency of oceurrenc 
of the parts of speech and then by com 
for the 


fourth grade as given in the above seal 


paring the findings with norm 


























reents ranged from minus 12.4 to 





: 10.7 pereent from the average for 
fourth grade which was plus 4.56 per 
4 { The rank order of the occurrence 
i f these parts of speech was then com 
th marks assigned to composi 
tions and also with marks in German and 
subjects. Correlations between .38 
i6 were found. These fourth-grade 
g s later went to the secondary school. 
(ft ne year of work their standing in 
r } rher seh o] was compared with their 
5 : n the elementary school and th 
found that pupils who scored 
; 4.1 percent to 10.7 percent (or 
above the average, 1.56 per 
100 percent successful in the 
school, that of those who scored 
1.1 percent to plus .8 percent 
74.6 1 t sue |. from plus SS 
: t t s 2.5 percent only 64.8 
a (Il t succeeded, from minus 2.5 per 
5.8 pereent only 55.6 percent, 
nus 5.8 percent to 9] percent 
62.4 percent and from minus 9.1 pe 
12.4 percent n| $2.9 percent suc 
The author sees in these data 
correlation between proficiency 
ge and success in schoo! work 
J ‘ Educational Ps 
XIX, (September, 1928) carries 
lhe t ais SSLOT of nature and 
d their influence upon intel 
ind school achievement. Seven 
h formed the nucleus of the 
/ 4 1928 program of The Na 


ciety for the Studv of Eduea 


Terman, Lewis M ‘*The In 

Nature and Nurture upon In 

Scores: An 

in Part I of 

; t National Society for the Study of 


Terman, 
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Introducing Part II of the 1928 Year 
book of the National Society for the 
Study of Edueation.’’ (3) Freeman, 
Frank N. ‘‘An Evaluation of the Evi 
dence in Part I of the Yearbook and Its 
Bearing on the Interpretation of Intelli 
t) Gates, Arthur I, ‘‘Ob 


ence Tests.’’ 


served Facts and Theoretical Concepts 
(5) Whipple, Guy M. ‘‘ Editorial Im 
pression of the Contribution to Know 
ledge of The Twenty-Seventh Year 
hook.’’ 6) Judd, Charles, H ony 
telligence as Method of Adaptation 
7 Baldwin, Bird T. ‘*‘ Heredity and 
Environment—or Capacity and Train 


ma 


NE 


W TESTS 


Classroo Procedur Test. Waples 


! Reavis University of Chicag 
Press, 1928 

Classroom Procedure Test in Math 

ittes, Breslich and Stone. Universit. 
of Chicago Press, 1928. 

Classroom Procedure Test in Natura 
SCLEM Cunningham and Waples. Uni 
versity of Chicago Press, 1928. 

Classroom Procedure Test in Histo 
and Other S ial Studies. Wilson and 
Waples University of Chicago Press 
192 


7 Standard High School Sj 
Scale, (Revised and Enlarged Editior 
Ernest P. Simmons and Harold H. Bix 


er. Smith, Hammond & Co., Atlant 
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Address all research news and communications to Editor Ear] 
Hudelson, College of Edueation, University of Minnesota, Minne- \ 
apolis, Minnesota. 
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My) 


"he National Association of Com- 


mereial Teacher-Training Institutions 


4 


announces the initiation of a program of 
leading to the con 


effective 


coéperative research 


struction of a functional and 
eurriculum. 
the 


of Southern California, chairman 


commercial teacher-training 


Professor F. J. Weersing of Uni- 
versity 
of the committee, is now soliciting sug 


gestions for such a program of research. 


The many administrators who by their 
annual reports show that they are vitally 
the effect of 


upon pupil progress will be interested to 


concerned with attendance 


learn that Dr. Carl W. Ziegler has 
evolved a concrete measure of school 
efliciency in terms of attendance and 


’? 


His scheme af- 


fords parents a quantitative measure of 


progress. ‘age-delay 


the importance of regular pupil attend- 


The 
College, 


ance. report is published by 


Teachers Columbia University. 


An intensive study of the Kinder 


garten-6-4-4 plan of organization is 


under way in Pasadena, California, 
inder the direction of Supt. John A. 
Sexson. The first Bulletin report, en 
titled Some Fundamental Concepts Un 
derlijing the 6-4-4 Plan of School Organ 
ization, is already published. Other 


Bulletins announced for the current year 


are: Administrative Organization of the 


Year 


Four-Year 


t Government 
The 
The 

6-4-4 


Studs 
Junior College, 
the 6-4-4 Plan, 


Curriculum under the 


Four College, 
of a 
Economtes of 


Seconda ry 
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Plan, and 
Four- 
these reports can be had by addressing 
the Principal’s Office, 
College. 


The 


Junior 


Guidance Program of the 


Year Colle ge. 


Copies of 


Pasadena Junior 


An example has just come to hand of 


& measuring instrument devised in tl 


light of the results of previous invest 


gations into what it is important for 
elementary-school pupils to know about 
the geography of the world. It is the 
Witham Geography Test, Forms I andf 
II, published by the Research Servier 
Company, Los Angeles. They are dis 


tinetly information and, while 


they for the most part deal with such un 


tests ; 


related facts as location, size, 
they 
social-economic 
The 


tests is based mainly upon the findings} 
of Rugg and Hockett. 





products 


ete., give some attention to the 


significance of 





' 
' 
isolated| 
| 


facts. selection of content in the 







' 


In the belief that teacher training 
will probably never be consummated 


educational institutions and with ampl 


precedents in other professional fields 
the University of Wisconsin has initiated 


a coéperative project of continued train 


ing of teachers in service. The gener@ 
aspects of the work will be handle 


through the Supervisory Service Bureai 
of Professor A. 5 
Director T. L 
of special se 


direction 
Assistant 

Allocation 
indicated in the 


under the 
Barr and 
Torgerson. 
vices is following % 


nouncement: 

















established to 


. xperienced teachers, graduated 
I University of Wisconsin and 
teaching in the State of Wisconsin, in 


fF or ee ™* 
PS FF SOS, 


y mit the transition ‘rom university 
“ k ft classroom teaching, proposes to 
a . . - 
the following services: 
* =: ] 4 correspondence service for 
~ ; 
teachers, principals, and super 


yy th i intendents asking for assistance. 

es of in 2. Visitation on eall to teachers, 

ie principals, and superintendents 
7 asking for assistance. 


service dealing with 
specific problems, the 
teaching of various school subjects, 
and the instruction 
4. Research ipon the training 
est of teachers in service, including a 


teaching 


supervision of 


service 





for continuous survey of the teaching 
" difficulties, errors, and shortcom 
. ings of beginning teachers, gradu 
su ates from the University of Wis 
al consin, studies of means of im 
ruil provement and the impro ability 
beginning teachers, tests of 
¥ teaching efficiency, ete 
while} 

Plans have been completed for a new 
nts twelve-story School of Education build- 
u at the Washington Square center of 

la New York University. The significance 
n f this to educational research is shown 
ings y the fact that an entire floor of the 
i w building will be devoted to research 
ia physical education under the direction 
! _ f Professors Jay B. Nash, 8S. U. Law 
ple ton, and C. W. Hetherington. Equally 
lis Spacious accommodations will be af 
ately forded the departments of music educa- 
rait ; tion, vocational education, art education 
eral mmercial education, educational soci- 
iled : logy, and educational psychology. 
afm The new building is a necessary result 
. ; of the growth of enrollment in the school 
Le f Education from less than 150 to more 
ser ; than 4,000 during Dr. John W. Withers’ 
snf— seven years as Dean. 
: 
Does the number of pupils electing 
a subject tend to affect the percent of 
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failures in that subject? The question 


is prompted by a recent report from 


Supt. M. G. Clark and a Research Com 
mittee of the Administration Club of 


Jowa. The 


interest all 


Sioux City, purpose ot the 


report is to teachers in the 


causes and elimination of pupil failures. 
that the 


The eommittee recommends 


study be continued during 1928-29 It 


appears from the data for the past yea 
that the highest percents of failure were 


in populous required courses and the 


lowest percents were in small elective 
lepartments 


report of the Special! 
Place of Science in 
now be had from the 
Otis W. Caldwell, Insti 


tute of School Experimentation, Colum 


Reprints of the 
Committee on the 
Education may 
chairman, Dr. 
bia University, or from the vice-chair 


Elliot R. Downing, University 


More and better educative material 


and instructional devices based upon the 


results of curriculum research are con 


stantly becoming available. One bigger 
and better example that has just come 


is the revised and enlarged edi 
The Standard High School Spel 


to hand 
tion of 
ling Scale by 


Superintendent Ernest P 
Har 


Director of Testing and 


Simmons of Elliott, Iowa. and Dr. 


old H. 


Guidance, 


Bixler, 
Atlanta. It is an 
collation of 


admirable 
example of the results of 
previous investigations and the applica 
tion of coi perative research to classroon 


problems. 


An attempt to effect testing econom: 
has been made by Peters, Director 


of Educational Research at Pennsyl 
vania State College, who has devised a 
test 


All marks are made 


cultural background that can be 


used over and over. 


In form 


on a loose strip of blank paper 





the test follows the suggestion of Dr. 


Greene in the February, 1926, issue of 


the Journal of Educational Research. 
The items are grouped according to their 
social importance. The test is diagnostic 
and it is being standardized. It is an- 
nounced as the Peters-Lozo Test of Gen- 


eral Information. 


During the current wave of interest 


in character traits among pupils, it is 


to be expected that existing devices for 
measuring personal attributes will be 
s viected to renewed inspect on. Dr. 


Richard 8. Uhrbrook has recently 


scrutinized the Downey Will-Tempera 


ment Tests. His analyzed results appear 
No. 296, 


Teachers College, 


as Contributions to Education, 
published by 


University. The 


and are 


Columbia report is a 


evaluation of the merits 


} rim atiner 
discriminating 


ind imadequacies of the original tests. 


Many schools far and wide are re 


porting that the Public School Achieve- 


been received and that 
Nation-Wide Testing 


The tests are de 


nent Tests have 
the Fifth 


Survey is under way 


Annual 


grades 3 to 8 and inelude 


sicned for 


eading, arithmetic and 


computation 


language usage, spelling 


reasoning, 


history, geography, nature 
health. 


supervisors report that the annual survey 


yrammar, 


study, and Administrators and 


is a spur to both pupils and teachers and 
that the lifting 


supervision above the level of guesswork. 


diagnostic tests are 


The first occupant of the newly 


eated William L. Honnold Lectureship 
at Knox College will be Miss Ida M. 
Tarbell who will lecture for six weeks 
on The Growth of Big Business in 


America. In addition, Miss Tarbell will 
conduct a series of informal conferences 
for women on the vocational problems 


women. 


of college 
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The report of the recent school build 
of Paducah, Kentucky, is 
Bureau of Pub 
lications, Teachers College. The survey 
made by Professors George D. Strayer 
and N. L. Englehardt proposes a feas 
ible building program for the immediate 
future and recommends a way of meet 


ing survey 


now availible from the 


ing the financial obligations of such a 
L. J. 


schools at 


Hanifan is superin 
Paducah. 


program. 
tendent of 


Professor E. A. 
this year after thirty years of service in 
the State Normal School at Fitchbure 
Massachusetts. He is now 


well-known work on Child Study and wil 


retires 


Kirkpatrick 


revising his 


continue to pursue his interest in the 


ethical aspects of sociology. 


Dr. Marvin F. 
Director of Education in State Teachers 
College, San Diego, California, has been 
Professor of Edu 
Education at 
Mary i 


Beeson, formerly 


Associate 


appointed 
eational Psychology and 
William 


Williamsburg, Virginia. 


the College of and 


Dr. Harl R. Douglass of the Univer 
sity of Oregon is spending a year’s leave 
of absence as visiting lecturer in Sec 
ondary Education at the University 


Pennsylvania. 


HE EFFECTS OF DRILL IN ALGEBRA 
During the 1925-26 the writer 
made a study of the effects of drill on 
The practice 
material was the accuracy drill tests of 
Schorling, published by the World Book 
Two classes were used, one 


year 


a special class in algebra. 


Company. 
as the experimental class and the other 
as a control class. They were not taught 
by the same teacher, but were given the 
same material in algebra, with the ex 
ception that the experimental group was 
periods from 


given eight-minute drill 




















1928) NEWS 


ree to five times a week for seven 


nths. 
the 


measured by 


| 

M . 

4 he achievement of classes in 
i rebra was Holtz 


s. The short form of the equation 


the 


uddition tests was given in October 
long form of the equation, ad 
wultiplication and problem tests 

in May 


third, normal group, taught by 


r teacher, was also given the 


Tests, but had no special drill. 


the experimental 


ged 186.8 months, with S.D 


r } months. The control group had a 


nis n age of 174.6 months, with S.D. 


normal group had a 


S.D. 9.31 months. 





all groups combined 

24.99 months 
of the experimental 
that 


8.D. 


group 
of the 


above 


more 





control class 


irly three times as large as the 


1.Q.’s of the experimental and 


i groups were paired to assure 


ilent mental groups. The experi 


had a mean 1.Q. of 98.8, 
16.4 The eontrol oup averaged 
es L.Q ot the 
The 
. ' groups combined 
McCall Multi 

used in 


Intell rence Was 


110.4, S.D. 10 


ups 1 nd 2 were ten to fifteen 
er in intelligence than Thorn 
should be to 

The 


standard 


estimates a student 


aigebra with normal 


the 


profit 


j ip was near 


estimated 
success in algebra 


Groups 1 and 2 somewhat re 


were 


4 arded in grade-school achievement as 


the 


sured } 


“A y 


Stanford Achievement 
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Test. The experimental group had a 
mean E.Q. of 90.7, S.D. 13.6. The 
nean E.Q. of the control group was 


had 
The 


S.D. 


7.6. The normal group 


106.5, S.D. 8.76. 


’ 


groups was Yo, 


The various A.Q.’s or A.R.’s for the 
were changing 
Scores, 


determined by 
Total Algebra 


in Other Subjects, and 


Ages, 
Grades 
Educational Ages 
ary iY the various T scores by the 
Mental Age T Seore. All Algebra Test 


totaled 


into T scores and 


without weighting 
various test scores, 


d a 


S.D. 16.2 


The experimental group hi: 
A.R. of 91.5, 


had a mean 


mean 


total Algebra 
The control 


Algebra 


group total 


A.R. of 95.3, S.D. 26.2 The 


had a mean total Algebra 


group 
112, S.D. 4.34. All groups had 


S.D. 25.47 


a mean of 102.6, 


e experimental group’s mean A.R 


f E.A. was 95.3, S.D. 16.2. The control 
croup’s mean was 101, S.D. 25.7 The 
normal group’s mean was 108.7, S.D 
22.56 Total mean, 102.5, S.D. 23.74 


A.R 


Algebra was 88, 


The experimental group’s mean 

Grades in 

15.1 The control 

was 105, S.D. 31.3. The normal group’ 
11.8 


The 
was 103.2, S.D. 25.83 


ache rs’ 
group ’s 


meal 


was 100, S.D. tota 
mean Pes 
The experimental group’s mean A.R. 


of Teachers’ 


than 


Average Grade for subjects 
Alcebra was 91, 8.D. 20.7 
14.4 
‘he normal group’s mean was 98.8, S.D. 
11.9 Total mean, 10 


Mean Algebra Scores for 


other 
Control group’s mean was 102, 8.D. 
2.3. S.D. 26.66. 


the four groups 


were 27.3, 23.2, 50.4, and 32.6, with 
S.D.’s of 6.76, 9, 8.66, and 13.27 


RESULTS 
The 
total 


experimental group had a 
27.3., 8.D. 9, 


mean 


seore of plus oI 
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minus 1.7; the control group had a mean 


total seore of 23.2—a difference of 4.1. 
The experimental coefficient was .46. 


100 that the true 


difference between the experimental and 


The chances are 89 in 


control group means is greater than 
zero. By the T Score the difference was 
3.5 and E.C. was .45 

The experimental group had a mean 
score on the addition test of 8.2, S.D 
£.16 The control group had a mean 
score on the same test of 6.2, S.D. 2.6. 


The difference was 2, and the chances 94 
in 100 that the true difference is greater 
than zero. 


The variability 


coefficients were 50.7 
for the experimental group and 43.2 for 
group. 

In the multiplication and division test 
the n and 8.6; S.D.’s of 
2.9 and 3.3. The difference in favor of 
the experimental group was 1, and the 
100. The of 


variability were 30.3 and 38.3. 


the control 


ans were 9.6 


chances 80 in coefficients 
In the equation and formula test the 
6.6 8.D.’s of 


and the difference in favor 


x 


means and 5.7, with 


were 


2.8 and 


» . 
2.9; 
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The 
chances are 80 in 100 and the coefficients 
42.4 and 50.9. 


of the experimental group was .9. 


of variability 


The problem tests showed means of 


2.8 for both groups. 


As measured by the gain, the ex 
perimental group showed a 12 percent 
advantage over the control group in 


total 
were in addition and multiplication the 


score. Since the greatest gains 
advantage seems to have been the result 
of drill. 

In effort, as measured by accomplish 
ment ratio, the experimental and control 
groups show constancy in algebra, other 
high-school work, and grammar-school 
work, none of the A.R.’s varying mor 
than 5 percent above or below the other. 
The normal 
of effort. 

The 


efficient of variability in total score of 


roup also showed steadiness 


oO 


experimental group had a co 


32.9, while the control group had a eo 
efficient of 29.1. 
Byron K., ARMSTRONG 


Lincoln High School, Kansas City 
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